{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Training_a _Classifier.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Eurus-Holmes/PyTorch-Tutorials/blob/master/Training_a__Classifier.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "eJheHHoNVYfn",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "%matplotlib inline"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "z8EYvp_1VYfx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "Training a Classifier\n",
        "=====================\n",
        "\n",
        "This is it. You have seen how to define neural networks, compute loss and make\n",
        "updates to the weights of the network.\n",
        "\n",
        "Now you might be thinking,\n",
        "\n",
        "What about data?\n",
        "----------------\n",
        "\n",
        "Generally, when you have to deal with image, text, audio or video data,\n",
        "you can use standard python packages that load data into a numpy array.\n",
        "Then you can convert this array into a ``torch.*Tensor``.\n",
        "\n",
        "-  For images, packages such as Pillow, OpenCV are useful\n",
        "-  For audio, packages such as scipy and librosa\n",
        "-  For text, either raw Python or Cython based loading, or NLTK and\n",
        "   SpaCy are useful\n",
        "\n",
        "Specifically for vision, we have created a package called\n",
        "``torchvision``, that has data loaders for common datasets such as\n",
        "Imagenet, CIFAR10, MNIST, etc. and data transformers for images, viz.,\n",
        "``torchvision.datasets`` and ``torch.utils.data.DataLoader``.\n",
        "\n",
        "This provides a huge convenience and avoids writing boilerplate code.\n",
        "\n",
        "For this tutorial, we will use the CIFAR10 dataset.\n",
        "It has the classes: ‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’,\n",
        "‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’. The images in CIFAR-10 are of\n",
        "size 3x32x32, i.e. 3-channel color images of 32x32 pixels in size.\n",
        "\n",
        "\n",
        "Training an image classifier\n",
        "----------------------------\n",
        "\n",
        "We will do the following steps in order:\n",
        "\n",
        "1. Load and normalizing the CIFAR10 training and test datasets using\n",
        "   ``torchvision``\n",
        "2. Define a Convolution Neural Network\n",
        "3. Define a loss function\n",
        "4. Train the network on the training data\n",
        "5. Test the network on the test data\n",
        "\n",
        "1. Loading and normalizing CIFAR10\n",
        "----------------------------\n",
        "\n",
        "# 1. Loading and normalizing CIFAR10\n",
        "\n",
        "Using ``torchvision``, it’s extremely easy to load CIFAR10.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "Jq5AS4P2VYfy",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torchvision\n",
        "import torchvision.transforms as transforms"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "fwxkmuF3VYf4",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The output of torchvision datasets are PILImage images of range [0, 1].\n",
        "We transform them to Tensors of normalized range [-1, 1].\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "wkyRUj_3VYf4",
        "colab_type": "code",
        "outputId": "3b9b2ab7-3069-4a8f-f0ee-bcf1d922f5ee",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "transform = transforms.Compose(\n",
        "    [transforms.ToTensor(),\n",
        "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
        "\n",
        "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n",
        "                                        download=True, transform=transform)\n",
        "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n",
        "                                          shuffle=True, num_workers=2)\n",
        "\n",
        "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
        "                                       download=True, transform=transform)\n",
        "testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n",
        "                                         shuffle=False, num_workers=2)\n",
        "\n",
        "classes = ('plane', 'car', 'bird', 'cat',\n",
        "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Files already downloaded and verified\n",
            "Files already downloaded and verified\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "6sudGX2hVYf7",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Let us show some of the training images, for fun.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "90MTXW6_VYf8",
        "colab_type": "code",
        "outputId": "7dd64bd9-58af-4147-c7b7-58668890bfba",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156
        }
      },
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "\n",
        "# functions to show an image\n",
        "\n",
        "\n",
        "def imshow(img):\n",
        "    img = img / 2 + 0.5     # unnormalize\n",
        "    npimg = img.numpy()\n",
        "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
        "\n",
        "\n",
        "# get some random training images\n",
        "dataiter = iter(trainloader)\n",
        "images, labels = dataiter.next()\n",
        "\n",
        "# show images\n",
        "imshow(torchvision.utils.make_grid(images))\n",
        "# print labels\n",
        "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            " ship  bird  ship  deer\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztfWmMZNd13ndr33qZXqd7Fs4MOSRF\naihKoSTKS2JtjmQrln4YihzDphEBRBAHsQMjiRz/cATkh40EdhzAccBIjuVAsGTLixjFi2RGjmLJ\nkriaorhzZjhb72vtVa/q5sc5551TXdU9PTPU9HT7fgDZNfe9eu9u79U55zuL894jICAgIGD/I7HX\nHQgICAgIeGMQXugBAQEBBwThhR4QEBBwQBBe6AEBAQEHBOGFHhAQEHBAEF7oAQEBAQcE4YUeEBAQ\ncEBwQy9059wHnHMvOededc594o3qVEBAQEDAtcNdb2CRcy4J4GUA7wdwCcDjAH7Ce//8G9e9gICA\ngIDdInUD330HgFe992cBwDn3OQAfBrDtC71QKPjR0dEbuGVAQEDA3z3Mzc0te+8nr3bejbzQjwC4\naP59CcA7d/rC6OgoHn744Ru4ZUBAQMDfPXzyk598fTfnfc9JUefcw865J5xzT9Rqte/17QICAgL+\nzuJGXuiXARwz/z7KbT3w3j/ivX/Ae/9AoVC4gdsFBAQEBOyEG3mhPw7gtHPupHMuA+BjAB59Y7oV\nEBAQEHCtuG4buvc+cs79CwB/ASAJ4Le999+91uv8o5/56b425xx1Lq3dc4mEHLR/AAAJR8cSCf19\nks8OzrS5nmPWw6fT6fS1dX0EAGi323GbfG61WvTveiM+FrXoWBRF5nw+r6XX6PDnFh+Te9u+ZXKZ\nuC2ZpM9dPQ3Pfv3rsFiofFuv36D7Hx6fjttKuRIAYGJS2+YXlgAAR47fBgB4/oXX4mOet8b517Vt\naYkUsOEh1bTq1SoAIJ1KAwBmplVpW15ZpmNFXUc+DZXKRtzmXJfGx/M9Nj6mY4loPYrFkbjtytwC\nAGBlTa/R5WVLJtP8vW58rLxRAQC02roulTL1+5899K+xFZPv++cAAJ82GqWnyY9aajas8PpFHb5n\nV/dau0PHHFp6jYiu0XC6th1Hc+N8EwCQSphFTiTp1pG2uQ4N1HW9aaPjXfAeTmg/ZBYir/Mhezxl\nng235YNzCXN+d+tZAzH63T/p+fdnP/Xf48/yvCRTybjtzlMnAAAzh3VPLs7T2jZ5D0+MKw84Ppyn\nttFSX98WVtf0GivrAIBai+fFzFUhQ/e/bXYqbpsZOwQASKd0n5ZrtD/WNjap/21dg1KB+jFazMVt\nUbNO36tU47YKvxvaHepj2/Sj0aLxVRr6Xlir0vkf+OhHcb24EVIU3vs/BfCnN3KNgICAgIA3Bjf0\nQn8j0G7Tr1LCSAQqCBjJhEXyLksXVhpPJpN9bS6W5F3fefLXivmdDkvXHZXiOhG1taN+CV3+yvcA\nACwtJBL2nqwNJI32kCGJLsFt3a5KT5ksSW/ZoaKen6Bl6na2l5CqlbJegyXAdkulw1SJxnzu7Nm4\nTSTzjXWSdJ95+hkzTprno8eOxG3ZVBYAMDutbRcvnAcAtFjSaNRUYxkZIhfVuq/HbZ6nIV9SKSvq\n0HdSSZJ4UmmVYLN5+ry2phJYF9S3rpE6a1W6R555mskJlcCqrEVY4bdQzGM7eJaavVNpUm7Vhba5\nBPUtyY+R1QZFoofTR6zraP4S3rQl5LuiZeqYEiyhd23HB0joXVbdOt0W91GPeX4murASOl9qgMTt\nRIs1z2OHB99xA2JWdghj2dhUaXVijPZCJmUkf9Ysms2m6QCNOZWmv6vrqoVtlklaXt2sxG35HD1L\ndiTFIj07yRTtSfNawBhrl+PDqn2lkvTtltHEa7yfa016HzRb+l5osfbXbGm/Ezw3ZaOxN5ptHh//\nNZp7k7WuZmTWyu2sAe0GIfQ/ICAg4IAgvNADAgICDgj23OTSYHU4mbTmEvrcTvSTNh0mGGKzCYB0\nbMIwKjLrldbkkmLSQ1RZaxqJWAXqdCyh2eS/aroQAjM2wxjVNxmTs8bkIqq0S+ugmTyLyVBDxKbY\n5JIw5JH87iYTtq0XCaP6bm4QKZQ195w8ROSSNbkIQXrhIpGdTz+pJpdSiYjJkaGhuO3O2+/isaj6\nPlQgsrLaJTV46cqC3vPwYbrP4Ym4LcME6ZUrGieRz5P6WygIyaSDGeL7b26oSanB8Qzrq6txWzbL\n5gxRfcvr8bG0TFtG91g+t/3WF/NHq2XNadwnb/YTE7BqhtFjmQQdsyRgu8OmnI62xYQ+fzfljHkF\nYq4ze4cX2hszXUJMIt1E3zExl3SdNRWxGcZvr+L3EKu8xwdJfztcAsWCmrWOTNP+2yzrOs4v0fq1\nO7re+Rx9x7FDhDdmjRyb33J5nY/hEu2dtHl/tNg8IsSjNfOMMJGZy6hZr8HrXKmpabAuZiCeB0sq\nb67TXq+ZfmTSshfM/uB7CD/fNeS2mFy8MW2VCtubAXeLIKEHBAQEHBDsuYQeNYlEiIwknRhADkiL\nSN5dI413IyZGUgMk9AEXSSZT/Fd/z0QKiXoI0Ca3qdQeuzXyH9EOACDLv8g9ro9d7pNxadsqodvz\nReLpRFZSGzCWLbCulWssud5x7FTcls+SZJJJaX+XFxcBAM0GrcFtx9XlsBvR3NSYiAKAzXWRbnSe\nZ6dJyi/niYgavf2Qns9ErRCnADA6NgwAOGzcJ7tdIZ9pvktGK2gyyST9B4CTx08AALwhlGQtOyyd\nzkwrKVopshRn1ipt5mErMvG1jJTK62H3pve9xHvK7L9MijUtp+tSb8W7OG5rx2vPmqeRBGWfDsqf\n1zbXEA3BMWndMdcQydJKmLKTvO/f/6L59mi2rPWkzSV8dwc2lDE5rq6mzRat43pZiVLRFKx0PVqi\nde7wqymdyPcdK+b1tZUX917Tt4hVshE+P5W068KOEan+tqG09qPI5Kk4YVhtTaT3lGFb06wFFAcE\nT66xdrmwrMS+W6c2b7Tu4tBw33evFUFCDwgICDggCC/0gICAgAOCPTe5NFtERPT48IqJwWh1iS1+\n6D0RoEygJDKpvvOtuiqmE1G30mmjdvN59UbdnE9qliUjRa1Os3mlkFdTQJbNAoMiRW3kYpdJoNh8\nZEgs8a3u9KjIQqxiW0QmCnJibBxAr/p39ixFfG5uqF/v+ASNdYbJy7e9Ra/33HdeBACsLK3EbXef\nPgEAcCY+YHWZzDZikpg6dXt87NIlIj43VlTVnJ6kvqUNSbdeIQJzdJRMLbPGHPPccxR8LOQrACTY\nt/vMXffEbdUKEVULi/MAgMXLl+JjBSabDhszzOih7dM4Z5iYdCklzpyYaMw61jh6UIjvDHQNEi0m\n+5225VLUj05DSfZ8hswksnVzJlpRIoMTpq3Fe8sltW8b7FjQYBNQx5LsvLHb9kGQZ8OYDMRPXfaY\ntXqm2WSRsv7tOzmgMxoN9cleXSPTXbmuYx8bofUeKmbjtik2yYkZKWdMIwUmsnOG3M6xE4F90OVj\nOt1vPsKA+JTYqcI8562mxJtwtKkZbocXxndsnAydUMiriUiuWyykuT/G8YPXdLOqpG8qdeOv4yCh\nBwQEBBwQ7LmE3jK/4gI/SAqPJe5+d0S0WYKNdDgiVTsjSYhrYoIjw3ykErqQPG0TtRbxL3AipxJE\nPkvkn0R0ppMm3wzfy5JjHSZF2ybSTFwvRTLvdoybWRz1N8B9bQcJPW2ki7EJcjlMmr7lMqQ9JI1U\ndvHCBQDA5CRJ6LMzKsF+/WvfBACcPHk0bjt+lCRnG5V66gQdj3h8h4aV0BTu9M133x23nb7zDuqP\ncRuMOjPUtxQNcOyQEqu4i3O5FPS6Eo06f3k+bkvkaHylY0Tsbm6qVjAxQW6TxZJqLMtL5F45MaN9\nExS5H5FRiVKSVyWp65hliSolj5GJzL1y8TwA4N67T8ZtsmfyRvUU19xhJvBGjbQqHntVE324XKe5\nT6eNixuzlamI+mi8ANESN1+v+0lJ+H5XYRmn1Rrleeka8U+eF7cDVW8jOoWAtc+tuP5Wq6oVt/j5\ni3gQiaw+o9U6a0SmH6IhW+k6djbgf2fT9r1Af/PmumPjo9w3nTiJGm2INmUj2eXKdo54WOWyjrnA\nxGqB3S1nD4/Hx+p8/cVl1YDbRsu+XgQJPSAgIOCAILzQAwICAg4I9tzkIilnBxWrtsm2ult8012P\nPzCpPq5liJEkm0usnWJLGtDImDqE4NhYURVIorlmZjUZVS5NKrFoZx1jShFiMmn8nUUVtKOLYp/x\nXnIUUCI28sb3vSvqqjXDbIVR41n1z5pouI02EY8PPvhg3Hbx8hwAjcYs9rjQUj/uu09NEqdOHad+\nmHuVOBHSyiKlyk2ZqMYhNnFkDLk4OUrmlEzWRr12etrm5ubiI0U2FRVMwq6JabrGeEmJTUlQ1WJ/\n52zWrgH9zReUwH75ZdoDJi2U9nGExlRt6f4QcttwuXG61Ywk4DLrc+g0kcOHStpv8H5LDGlbXdKy\ntkhVb7RMkrU0JTBLd9TkkmPitdtU1b7EprUEmzXsLqlx8riEJd5l3xkzQpZZWSHmmk01H0Vsrmkn\nje+7s6S99K0XnZ5T2M/dPNPiPLC6obEOh9hMEffDRGlLJKd9L+SytIL29SDmoiw/hyPG1JZNi++7\nLmTSiZlJr9Fm8rvGJhf7jEocRi7db26tG7PJ4go5IBQ5gZiNgxC//1pDd2DX7/R87w5BQg8ICAg4\nILiqhO6c+20AHwKw6L1/M7eNAfg8gBMAzgP4qPd+bbtr7AT5tbVFHgQ9EZSDwuWkj15ShJqk/yzV\nWim8wxJBm/82TI3TiH+JbX4QSXdZM2lAl7m4gkQmHhpXomOECzM447boenKyEOJ0pwPGLNJ7x/xa\ni+RgI/u24u1vVcn71IkTAIBXXnopbstyRN1b7ntz3HY3k5X1Go39ySf/Nj72zgfeBgCYntQ8LCLJ\np9MqDm2wG+TIBEnNzqSXXVojbWfKuJOOjJBUbUnLep2kzfUNarO5c4TcTibMVpUUyjbSl3Pw5Nlt\nrGuje3n+WjWTvrTKWtKA4LzhFM19zkiCVS+alnGNlRwhrAU2zV7rdkkaKzctscpuhS0lAZPcbymC\nsFLXiMRUlnPK2FxCMr/mcfA81izvq4SJgpXtVzWSY8RdajlzXdZs0KJ9kjFFOLLiYNAxUrvs4R2I\n+l5PSXYYSOv+kJxHlbpKqYurtJ+Gh0k7qdT0mKSj7Y3Epnmz+WtyIpmztN8wUnCO77+Wsto/rWm+\noCmrLy3SXlxcJs02bfotEah5owGneC+urOv7o81dmuFcRgmzBstcdKXa6M8ddSPYjYT+OwA+sKXt\nEwAe896fBvAY/zsgICAgYA9xVQnde/8159yJLc0fBvBD/PkzAP4KwL+9ng7k2N3M5i4Rd6ZBOVTc\nDnleEsbAWWWpb5nLrAFAhTO9tbg0Vcf8Yoq7lrOGNA62WFvUa2yYQBsAWJ3QX+R777+P+mHdFqUY\ng4lMEMlc/loXMcnJYu1pvtufe2MrooZeY42linlji5ZcKOfOaUm5BC9/eZM0leef/U58bGqKXAmf\nfeZZ02+SIISfALS0WCZDknG9qXO6wRpQZl1tpC+//AqdZwK4pEiI9O3MmfviY+2Irr+2ptkTvRuw\nF3h6L7xOwUx1kzPkzL10vSMzs3Hb4Slyt5zv95pFl++ZMlJqnu2l1iWw66WAAq1Lo2vKzfGubBst\nbHSE1IHssLHvs8V7eZ3mo2Xyx6jrqi2LyC2GG5LsgrG2a7oh5zvrohj/NXtMXGm5LZnSeybEHdfk\nzhG+yCW3F9GtrVts4qkBOXRaZlKXWBteZ02lagqmiLuvHxCIZ90nkyxNz7PbpA3IE97Dtk3M0zM8\nZUofbrIEvcQ5V8zQkYqle1O0hvsxbMotnj7FuZH42OX5xfjYhTn6XLbuzHvotjjtvZe3xTyA6Z1O\nDggICAj43uOGSVFPovO2Bm7n3MPOuSecc0/UjM06ICAgIOCNxfW6LS4452a893POuRkAi9ud6L1/\nBMAjADA7O9v34he1rGtqKQ4iQLe2WXNMhskJ6063ukJq1IZR1SUVq5ArVsWRyuodc90oEt21vz+S\nknNuSQs6SCTdm+69V9v4u51uj65ObayO27HI/W1t0zhNK/oJVkHLuJldvEQFKy5fvhK3Zbi/G+vq\nFtflW6xyrpVaVdXb82cpijTqKlEzM0tmmGak5pICE0nlTSKzsjl1EfvB7383AGDdVGQ/+xpdN5PV\niMi0VGK/jaJIkyZPiXy2boixu6fJmSp5Q2rs2tbpUcvp74YprmCLoWxFMtWf20Puad352kxuNnkd\n0z01asUMom2NGu3FwzNKpJdYRZf5SKZtbVjqR91UnF9nAnthXfe1kG8Jls/aZuySeibyOpbIS5EM\nQwz62DZD1zD7L+J57iEj2fwRmb5tnVEpXAIMTssbw4iVLd7rkkMFdi8k+tNIO7Uf9SG+l7lnJCYt\nc/5ahV0fvZpPpXDG1CEyk9UNYSkm1c2yPi/FgtTb1bWd4bw0kxOUhyhO9Qugxi7Wmayud9TpdwW9\nVlyvhP4ogIf480MAvnjDPQkICAgIuCHsxm3x90AE6IRz7hKAXwbwKwB+3zn3cQCvA/jo9XZgEDEY\nS+2mzW3JkmZd/oSYW19UqWWepdOWyYMhopo4+BeyNuCA2jIm41lSCBTjHieShhA09aZef2mBFJWZ\nmZm4bXiU3PS8/fWNU0H05nShLor0ZAmr3kyTgzA3r5pCs0mmrbpxB9uokzQ5PKR5UlIc8FMolPhG\nKi0sLhD5OzSirly1Mmk2mZzmEUkluBBAgolsE4yztMbEjwkcabeE9NV7SSiMuBzOXVYSenSUpJui\niXqS6vbO1N0TSfj+t5C7pc2xs7xMQU+bJgdNHHSV0iyOgqZoboZd9NyWN8EhhRL1V7SDujEpHp6i\nkmtW+3IJul6zpvNRW6c1WuJshPW6ni8l4joJk3OIfQ67RmKMWDbuSGZAI35GcQZGK42LxqdtIn13\nmHDuJNR9UmLcumZPCinq/fZS5cmjmhtIcsVYCd3mTumHlHPsL9dncyVJYQnDv8YCuR+QnlTIWW+I\n2DjIyBS3yfB3D03Tvpud0X2S5WC3v31OyyiuVEgLXVhQY8XEcJ7vSXvNpPrBzDi5Ms5O6btCnDVu\nBLvxcvmJbQ6994bvHhAQEBDwhiFEigYEBAQcEOx5LhchAa2ftqhl1o9167GeXC6SytNocFK0IWvy\niIj/qmcVP7JV3fmYJUpb3NayEZ1MFkkEY84UuGhV6Nj6mpKAhVKxf3xbf0dt1J+osDbPjBA5O+TP\nrVaUqGw2yZe3aCLfOi3x61WzwNFZMgs4Tyr91JTe88hhSvs6P68pap96nHzSpw5Pxm0lTpcrZqyk\niahLcl4XW5wiM0LrUTYEZYa/W6tJxKgW4ViaJ/NLqVSK28YmyIxVGlEzzDD3Q0jZZl3nQ0gsazqL\ndsibIaRoNm1qlrLVo13Xfq8v0zxfvkLmvfPnzsfH8rwvVteUaFvbILPY/MWzcZvUbF3ZoGvVTQGI\nFEeIepMq13H+mvFjmpb3jjedAaBRpCkTwSjconf6qGeyfL2uKZzBe1wKaDQNKdqOpMiD7o86z691\nIviBLTVDJsbUvNdfTVXJfhv64bfscW/MR4N8zuNjZjnlO/H1bY1Vya1kzJd1IdlNRzL8MpmeIrLz\n+FHd81k281ya1+fr/CKZKC8v6v6otsmzuzRE+9qawuRWzpiFIibc33X7XX3j2y2ChB4QEBBwQLD3\nEnpbJPR+AhQmCk1+uOMWc/4qJ4lfM8niN9itq2lI0TjLoZSQMlLz1gIaAOD5l3gQuSKkaMLIHOLC\nN2Mk2DZHRFo+M8kkl5P7G2lIRCpb0R5xwY/tSSTrIiZ5MDKm5FWO3eImxoykwZJaxErGcEnFnMsX\nSJqsVlSiF9KyUVXNph2RNF3gcndRV4+l01IAQkm9NGtMm2UlBnN56pu4wjnjYifkVdVEfm5s0toO\njajUfttttwEAKnyeJXNlmrOmUEmO3cxWtRsxnnjqaeprW6X86jJJW3/ztb+K28o8NxubJJVZrUMy\nXlph8vAskYTlVSWwJ7gU3ukT1P9sVsc0zNknE1mTx5ALrLi8nlcq0Tx3WT571UQDd8QN0EqwTJDW\natpYkTwpSZqXyLwa4vJ1PblzuKiGdUPcIqGvrlSxFVbr9vzdyDyH7dh9WFwl9btJ3s/WcSHF5HfK\n9E0keCmT58w9hUfv0RTkWFL3hySilK5JXh0AOHUvRYCO3anlFtN/SXmTvv0NzYeU4RxG8hwMctm0\nmnszunH5OkjoAQEBAQcE4YUeEBAQcECw5yaXFqe2tCYX8fXuGpNLl0mKrLBThqh85QWqUF/ZUD/0\n4SGpKK6mCPFfzbIKaxPOi39q0kQQJljFS/Uksqc+ie/7piFAJbFWymhWkpbUJtbyWwg5Wz1cUqH2\nlBlnvXMnP/TIJDeLIo58g6qQQ8VhPqbz/Oorr1K/2f914ZJJZLZJ/R4b0/S5EYckZo3/vhDMkvSo\nVlZCc5MTTuXSav5wMRFm+stzJORos6pmnkq13HMMADJsoimbqNfnK88DAE7fdSf12yRaWq2TKa4w\nrP1Y25R+amRffGydjk3mdM1GuDjGu79f0xSneR6GR4j0FbMTAGTZxJU2hTnSnIguZ6Ki5y9fBACc\nuuseAMDE9OH4WJ43kjPmhDrvD7uD6rwX62xevMCRwgBw/iJ9fvWs+ky3eB3rbWfOI//pGl8/nTZR\nnmwq6BoTnmcbTi5pI3h7TSxTU8bHX1IeG2NHnUnA9XUt1tGq0jUabKq0/vNZXvd0Sdcxwz7hWRN5\nLD7vmgTP+tQLS6z9ELONdVbI8XuglKDzh0zxkvGcOEToPdGlPZsv6LtCYiHE4jPIZNoxnhxRcvvn\ne7cIEnpAQEDAAcGeS+gSSWcj6kRaTidM+S6WWBs1+gWvlY373VFKhVriivIAMDRMEmk+p4RSgiVt\nn41/MvUY/4wmTWRakk9I7FDZ/MXnvht/XpgjFz9LjpU4ZWqPO5bvTZNpyVnRVDq2MIekDd1BQl8x\npfNKHMFopWAhZ0sFregg0k2XIyJHRvVYPiNRkCaHBa+HTU1bGKLzmhwt2THFGLqcLKZWVQksknSh\nPXNP/5iYJGm5a+ZHcrhsGBJ1KEH9TBpybGGRiMZlLlDyvh9+j70B3dJIuhkmSJsD0ufefgfto9PT\nyvLde4w0lZQhsM9fIHdFye9iCT9Jfds2bGRb0iBHOkdNdlNcZre3rnkkZ6ZIy8inVQrOpZiMN9pr\niaVTz+Xujk6qZPzO+ymv0KYht89duEQfUiqFX5wj7fabT70MAHjhFXVXTTCJ3zX96LIUnHHb58Sx\nxSlKTEIXC0bTStNYGhWVoDdbtLeiNi1MxURXLnFRFLeo85xmDSFjNCGJKI3fLUZCl2fCSstyPkxJ\nvrEiXTcHKj95+qg+G0vniCCvmfdTmYuztKzbM+91WSlrhZBIcEuKtvYwfW5AQEBAwC2G8EIPCAgI\nOCDYc5PL1uo9gJpcrF/5hfOUdlWqDklCJEDrZRYLGlFXYD90659dHCIyJTdMpgYbfSgpeJMm8Y+Y\nSXr81UV9ZzPB0Kiq5ZeYjFpZ1X4XmcDJmoRWXn5HJRGX9X2XQFHDenW72ydAElTLatbocOKmrKnl\nmeHox64x5UiFp66kfzXk7xqnek0atVIqtdTqqr4LJ5Zgc0LSXKNZITNJq24JXplTE70XE3F0sUZT\nry+1HA+NqBmhycRuuaamn3SG1nlpmci9J558Oj729nf+PTrfmB2SO+z87zz3FADg3LPax89foP2X\nMuRzjc1LEoloCXUh2qxZqMNJ1tLGXJLi/ZR4icwgNpo1l6frFcy+zmZ5rxuyf5jNekM8R7m8OZ8j\nVkdHNGrz/jMUWWqNJffeQWN497veCgB47fzF+NjXv0lzee6S+s+X2RTXiLafyA1jmrvtFJkuvu9B\nrWk7PkrPxsvfeSFu++pX/hoAcHaBTBgNE9fg27SPWm1ta3DVKDh9H8QmUnm+YJ8f+WzrFfMRYxYV\nXj7L+29yTJ9zcapYNsnvXjtH89WOlCiNTTlxwj1zT34H2ORmnRvnRIOEHhAQEHBQcMtI6BZS3OGZ\np56J214/R25XNWaxWiYPS4aJs5JxGysVWRo3krFEbpWKJOUMGQm9yOcPG4l7aJw+F815IkUm2J3K\nSqRCtNVNHpFFTms7PjFhzqM+DUw9KhJsj5tjvyS/FUcmtApgs0n3lzSfgLpsOlPUIMnS4fghkt5e\neunV+FiV53e4qBJHpUrufKOHVNpbY1fRFrtiFobUpWyDI0qjhq5VUnKKGGlI3CaXV0jLMOlgkE0z\ncWvcLVfYXTFtoiVFOs5kSUp9/YK6YB4/Qf0uDRk31QwXGDC8u+AjH3ofAOAbX/la3PbcFVrH8orm\nZplfprbVNdLIelPDinurjl2KQdh1lPPUJdUek9qp2jfV1kwjn5BjSd5qpaJ5WsJ7nFMS33ZE09se\nve04AGBqmtqmZvTYfXfQ5zuP6x57fZ6Iwe+e07q1W9HuKsk3wsUe7rz3dNwm6XULZr3PPU81Zxc3\nSfvpdFTKlymqm/3UllqoVsvlv6JF9+ZOklxQ/QVnnNmTHZbkL63Q/v6b5zT/zibff9NoiPPzdN7E\npLqdSlre+DnfoSYwACSTNy5fBwk9ICAg4IBgNwUujgH4XVAhaA/gEe/9bzjnxgB8HsAJAOcBfNR7\nv7bddbbDoCIPIrVLgAcAbFbo13Btk+yyTeOKJKW3sqZcleQdsXbyYf7cYOf/jRV1LxTJO502ZdvY\nDaw0rNeYmKBcKIenp3u+BwBjE+R2tzinLl9r7EY3xIFOAFAcoutF0QAbOrbHjhL6UZUMNniOGkZT\naLG9N2lcAkdGSAMRW5+4IALAOts/KzWTn6RJNsNyVV0IZT2SWc4nsqZSTpnXTFwgASCVkGALnbds\nmrSAmhSKqCsfUGuStJnLmcyALbbfmumQde7wHFU2tN8vvUiueO9+7/fFbYkUZxc0bnGCWc7c+NMf\n+0jc9o9/9IcBAMuLKvnX2dZh9iBtAAAfE0lEQVS/wmu8uqrS+yZnjKxUVIpbX+fMikbKX2eup8Ku\nnbaUoJTTsxpfsymBeKrZSk6ebsxHmUyJdXpO5soadDd3gZ611UXN6/O///zP6INkOjWBXHfeSdn/\nbjdZAGtxVkaj4pg4GwBwRgMtc5nDlTl9vsZ4W3QaOr4sa5KZARlX88wf2OA4MUv7nnSL/Xlg4j7F\naR/7NWBnbO1yWoODsF66pLzYIpeeOzyu2vzJ45TfJZ3SeVONrV/T0nvr52gHPmK32I2EHgH4Be/9\nPQAeBPCzzrl7AHwCwGPe+9MAHuN/BwQEBATsEa76Qvfez3nvn+LPZQAvADgC4MMAPsOnfQbARwZf\nISAgICDgZuCaZHzn3AkAbwXwLQDT3nthROZBJplrhrisWdVKcmJMTukln2XXpgZH2XVtSlupQN5Q\ndbVa5Qi8FTUPpJl0yHMuhpERJYoOjVFUnnUHy3G0Wt3mFmG1WfKNTM5oHwv83ZFhve78JZoia/4Q\nclaG3DZRslpFffvo1EFI5ZRZGsuRKthsqKlD9D0/IE2xrMHR47PmfPpz5aLmBYkimo9qzVQ756jU\nZqfRcy1AiyVUTA1NSX2aNOPLsRtYikkkKXQBAM2I5i2f7y/G0DSBdRGT5U02xzSNa2VZ8svY2q07\nROUVOFNK0ujDmSx9PnxEc8QMDZOa3eZrtY07nXzTRh43m3TdRA+fSf+Q+q+WRK1W6/xXxxKbX0zf\nJEp3hc1BVWPmkfOtm2+L3VprdT1PSEBJadsyZptB5svDJTJLvXJBa2huxbHD6gjgmKhfel2J92KX\nTE+bK2qpTSXFNZHnw+yn9sCIaflszTBCQg54hsSj0T5fvt+V0cXptOXfenqpSPv19pNH47apCXIU\nsPtfopwNVW4+9RfqaZuapteLXZOizrkSgD8E8PPe+54s0p5meKCB1zn3sHPuCefcEzVTLScgICAg\n4I3FriR051wa9DL/rPf+j7h5wTk3472fc87NABj4U+29fwTAIwAwOzvb99L/zt9SQvjjx49rp1h0\nvftOJWG+/a0nAQDtVfo1T9oq5syM2PwW8mtof8w7LKFV6iQpVRvL8bGFZbpuzhQTGOGgoIlxK8mT\nlCKZ1krDen51lYinjOFnchyY06hqEIL0NyZjeuty0Tk9+V62L8kXw6xkW6Q8k/YxLk5hNAUpRyfE\n7tiEjlMyQnba+iPsuiT5Hzmiksmd97wJAPC1b1BAyHMvaJDIvW+hY2fPK5H46ssk8RvODVUm7vJM\nfN5116n4WJcDS5yz7q00rkuX9bqLi7SWEvxkK6gnWFKzRRASHBEVDSBFWxWS6NMFkweIybp8yZSD\n46yJGc4NZIOCRFpfXLQEOWkeVlqWPSBaqRCcAJBwtMesJjnBxLvNLuil2MpdTAhXTO4clhh7iivw\nx3ZXB99i6VBk1UZb92uHJdi0LbSRon6+8vofYzscGtM+tliDO//aOe13hYjGjAlxyqTF1U+yIhpn\nCck0abOTxkE7et/YwUIIU/N8aQlLbRO3QpuzyfNDmcvRPpmZMXl97ibNbOawyX3ExUh6skPKWMTx\nwwb18bNsnSoaTc6CievHVSV0RzPwaQAveO9/zRx6FMBD/PkhAF+8gX4EBAQEBNwgdiOhfz+AnwLw\nHeecRPr8OwC/AuD3nXMfB/A6gI9+b7oYEBAQELAbXPWF7r3/a2zP0L33Rjvw5BP0G3HxdSXfxPxy\n5r7747bTt1NK028tfxtAb+VvJ2YK1x95aVVNNVkMqhpO16jXjO82p/+0/sXj40QGCd+SyaofbpJV\nqvqGqrxDrKLnC3peh9VbUR170moOyjWxJSfEIERGDZUaim4AUVQx0W1SpGOYIwcjQ8qcPMVq5bQS\nWzXOo1MqqU99MkV9+oEffBcAoBGpieb4SbrG5PSRuG1hnkwj73zH2+K2869RFPDSEqng73zwgfjY\n8BCZIpotve4Q339xSX2Dn3iC8o1cuUIktDXhvf3tlJ+kbVL7ri6ShXBkfAZbMTZFZg1LVq9yRGwi\nZWpXskq9uEARowVDqIsJZcTkoOl22Fe/qySg+KRf4JS2CwtquXRci/XYsWNxm5gYrNmhwOa0QxxX\nYCOg5f49tTx5L+SM43iD94KYlqbyal5xbB+rd/S5+dSnPwcAeObZV+K2mXcZUh3A61fU5zzBr5rl\nJb3GxgKZto4e0v62OQqzxCbNDePoIPVGLSma5b5lnb7KGk06XqtLrh3tk7wPkgn7/uBoXfNayMem\nFiLBj5ko2TSTnRXjPy95hWw+FilckxbfepNbSdYjMmaYcpXMUpOlk7hehEjRgICAgAOCPc/lsr7O\nEYkVJYoWF0mCWVvTaD/5RVNyr5/N2imSElBJWIX2/twbPfwkS8be/HSXOUrs1Vcuch/V4eee01QF\nvG5c5hwXJMjkNP9Jh3Nc+AHuUnG5KuvbFud32d6VcWNVIwFzXNRjwuSPuczSkjcSQY4jNLuciL9o\ncuFIRsW2qQAhc2/d3SJ28WtxpN6Z+87Ex9Y2af3GDqmUOjFGn+vGte5HPvh+AMCf/DHx7c889WR8\n7E1vopJyhjuKix+Y5IY4c4YI9MkJklIPmXwzQqhumhKFmfT2W9/zhdMm8+ZUhiS0yLjzyfqtsSvr\ntx/XfouEXiyqNpPLSNEVbTt2lD4fPUIaxcZx3U+iQVlto8oRpVJuDlACVkoqNg3ZObespKzpOPUx\noxJ6hiXyfI4cAZwZO3iffPmx/xc3SSGMZte4xm7BzJiW90txThnrLtrhW1Qj3deSX2iyRPe8Yt4B\nY5wPxq5LkSOUjeKEDXbzvLxIGsC6qZLh+VkzwjK85/XO6HXvPEGR13feTg4A42O6hwt8T9uPrhdt\nWzUnUbwzvHltKUtxda0ZF+DkGyBfBwk9ICAg4IAgvNADAgICDgj23OQyOkykQ7miqlV5k1Sm559/\nMW7LcRELSYfbbKjqFpN/V0kQr2lo+XuDAsksicpFCnKmLqlLeO6vRLIpYTo9TsmOkjYlcEPIEu1v\nQsw7rF7abksCLFtfMMvJraya3Tc247a+OEfE2vxlVbenuZq8EDqAzkfEEYzrTU2GJhafmjGNZNgU\nYQnYUVYnxRRRKKnZJs0+1Qmv95xgP/6lBSXMUgkiLe8/Q/Uvn35aTRfHj5DZaHpKzUfi7p0wKmzU\nog4fOyIkp4nAY/IqZ4jpPPtUD9oyZzk6dskQlKNDpO7fcaqfsDp9B5l7jh5V04jUlV0yxG1lk9bv\nlVfUF/vxxx+n8znJ2cioqvYdrvVq67pKGuQhQ0znmYwtcYpcSSAHAMOcFE4ioQGgxKYzMbkBiO1X\nQgSnTEzHl//iMQDAH/4vTSfcBs1fMr29TPimU0o4y15r2mjudTIfLa7pHnPMYA5xGuaZCT3/zH2U\nendkSPdY3EvjWCDB1hfnaT8//qxGp6bZNnPymO6np79L6zFskvC9/T4y9d3NY7DmPamBmzJ2HnFm\nSBryWRwtpLBFb3plOq9j+r2ySfOgT+G1I0joAQEBAQcEey6hlzgasye1gkRQGjdESRvg2D3JRuWJ\ngN5TLCPOct9/z53IU/sr2uEoxUZLz5dCCvIrHRlXritz5JI3a6quS/4O39WfeAq8RRzeZkuXrXCa\n03WT/nVmhtzBLl/SYgIFQ6wBQC6l5NQ0EzrrJv1wMUvSh5Bq1A+JViPJ1eZcyea5JJ+p1SZ5Zqam\n1YVLcn4sL5Omkq6qtHWISbFWU927hqS4R9GoFHyNt565DwCweEVdWBuczyRjSwPylNsSe44luhq7\nflkNp8ptHtpW4PEN2glf+NL/BQBceE1d8krsnvr+9/yDuC3HlexFu5SiKoC6s6ZTGdNG63706G06\nFibcJS3votEKLs+Rc8Cli5fitjJHgaZNaTsh7x2vo3WfFJfUU3fcEbcdmZ3mv1rEYmSc1mqIyeQr\nZ7UE3R88+mUAQLVi7uklD5CdwQIsSiblsRR7qBuyv7xK+3NlU/e6uCEeniKNYuI2lfLv4jJ29tGP\ntW1z3zb3KcXzsbSqe7LI0b/336va1MUr9NxmTPjy6HCR/9KYhPSkzxKZbiKPxZnB9CPmR3mu7PtJ\nIni7pi27UyT4LhEk9ICAgIADgvBCDwgICDgg2HOTy/IKqZo27WSOa3PKXwAYHuZoRlb7bZ3AOic7\ncsZfXCIj3QCbixvAhtpoTYFnH+uu6VuSTScprkySMmpai6ubNGyVe04HurGhxFYiuc79oH+njFmj\nwn7uYqoBgPU1UrPXVlU1Lcz2mlwS1leea26WCmoCkKRcmp4XSDGhleZET6mWjiXiqDwhUwGgwqpx\n3lw3yT7Qo0O0PrbeqKyLJYqkoMuRY6ruuyRdY4RL2Jy5/974mNSStZr9plRJMv7wsh/ybNLJGPJX\nEixZ0ktS9Q7IzYVnn3sNALCxrMm/0myuWVz8Qtw2xOq4VFCyfv9DTKI2DaG5tMiVjZY0KVyX1+PI\nDJkWIpMTeLRIZgd3WPfHZhytq2swMkJzfv4iEc3zS9rv9RVag8uXlLxPpnvNhgCQ5etlud/1SPfJ\nZpVNDAmTwpiTd/VKhL0mF2tBiM2nJhtamp+vTMaQi0J4s6/3mIkibXCK4XbLksScjtmY5CT6UoY3\napKbiQOFDUMY5nS4FUPYrvM8b1RpX9vIUok/sKmRxfxi0x93Or2ViqwzgSTqciaZV7XKa68W22tG\nkNADAgICDgj2XEKvN/oT9kvkmyUvpXq5YymhWNRfboki7VZMoQhJndnDtsoHiRjdWVKXw0mbV4U/\ndn2vSxIAtFjasr/0Epl2/nV1IdxgKXxQOtxIxm7aKpv03Wple7fFyEgtFSYmL1xQklM0iQlTNCTB\nklqbyV8biCpFE+bmtd9FlsxbLVtIhO41PkpSYjatJGCL3SwXVlQirTfp/CPHVfJPZniOLpNk3PQ6\nzpUKEYMNE+Ha4pqSaSM1Sb1JKXBxZFbzimxwfc/1DSWJRUIvqFAdI8GSl5W4XUTXXbiiZOHSMvU7\nzfPYNPVAi0VOwWtqTIq0vramuVwuM+H5zW98g+5tFiGTJG2jE9now/69m+aIzy4/zs4Qpll2Ucwn\nVXoulUhi7bSNdtmgMaxwZGbHSJ8ZzquChHGl5XtY7XIrGmY+YldT4197qJDm8am2sSEOCFwfuGWK\nhpQ7vq8tneqfj44QmJwC99CoviuqTCp3TL7dcY4url1SzabRpHs02735WACTDtfmx2lLjVrjPilp\nvXmfds27QshkWxRlvUyOH/kgoQcEBAQE7LmEHtvCjXQoUkizqb9ebf4FVJuZcYlig1gxr1JIhXMk\nWNurSMs2C+FWWK3AS44JIzU5/qGWDG1WQpcyXnVjc5ejUUXzu2SYG5Cx2AgnybXirIze7f+F34qF\nBZWkpYxYaUhthzJfoulQ3zkXiRQNMT/vYh+smmIJbZbMG6bylIxBpGA7fyJJ2VIdhSJJjGmb14Kz\nPIpLXtbYv5PcqXUj1U6Mk225YnKzNDvMEbCv2MqKBvRUOS/N2rpe4xC76fVafQnz8+Q66JumRFuT\ni4GYQhsipYp7Wq2q7plSUm7DuI7WWMOyUpTMVxy8Zut4sGTe7ZjSZK7fBa5T5zKOSbpGoqszXua8\nN9YltcKSYCqpfEdX3HE5GCw3ZFwwWevqdlULFM7L5rYBTAEMAIurundkuUeNq2k2R7OfaZhso215\nbmleGi1zfbare6fScmtAWTqxcUf8vCRNoRckJBOj7uEpltDX1pWjkvkt83NrSzdGcRbFtDlfghb1\nXnV+fwmf541WkOSgpKxJUmSUoutGkNADAgICDgjCCz0gICDggOCqJhfnXA7A1wBk+fwveO9/2Tl3\nEsDnAIwDeBLAT3nvB3mBXe36fW2iPdkISlWpOP+IKVYgJNCoKSbQ4fMlafwgDHJVHIRBZhWxllhV\nrMsqck+9wm4vcQYAQ0NidiB1y6qL4s6UML+1cYpNv31/R0ZVfY7VYUOmCQmYNLatoWF2UWNiM2MI\nrvV1MlkkjB4o5iBxGwS0mIZELnZNHzNsRmiYqvWSi8fWUpRUopJ6NJNWdb/AaV1X19S8Ms5uaDlD\nwOb4vE2OhJ0zJqhhdqWcOqJEaTa3fdrXOpPa3pB6JTZVFXMmF46TGpc05nLZpBWWnCjGNCiudTaF\nbInT61ZEZXc2mpBU9mTaPgd0r47Zk+JSF3PE3h7rL44ikbvthN6ryMUxhAAtVxbiY6hyJGpP+mY2\nZ/SQoqOwsGlrC3k6L2/MCo7noW7MKl30Ep8dYzON4lq8eo2YcDSVJVpR73XLVb3+UIn2SSqp5w9z\nXeBSUfeEkKLr7KqbMO8i2aeppDF7Sb4W83zFrtgS+W7EZyGTU2kzmIa1t10fdiOhNwG8x3v/FgD3\nA/iAc+5BAL8K4Ne993cAWAPw8RvuTUBAQEDAdWM3Jeg8AGE30vyfB/AeAP+E2z8D4N8D+K1r7UBc\nWML87EpGuUHZEEWqTvb8Ykr1dZWejh+nsl0vvvJa3CaSlBB+PaXfBuR30eyM/W5j4m2UNlnpIK51\n5roiRQ6bLISSLF+kp64ldLoSFKFjqbKEW66oZLwV1pVLMDSkwUez7MbXNi6Hi3MkxY5w5j6bPa7A\nktqSCVKRABobWLS6TgErkrfFuhJ2hIQ27l1CyloSt8TZ80R6r9eVXCxyObXVVR37HBfr6LmXVG5n\n9i2V16C0cpXJrqZu9xxLv4dNwkFBgkm3jJHiW0yKtkzfPLvxSQEDu5868T41rpWssQiBCwARk75C\nitrrd5nM67bt/hMt0JKAvQFwdie7tLgy6hokU7wHnJKcLQ5oihI0zqTRfhIJmiTXsW38jO6Q4rRs\nyjl2uZxe1syHYwm60bN36Xgr1tpUum7K/PZI6FKExrg98zxUmCyuGAm9OETzvG5yDkUdKXCh+0Pc\nFTeqNEcFk6lT+lQ3rsKJhDyvtlgNv9t4rRLmhVbn/dcwpLJoFAO25K6xKxu6cy7JBaIXAXwFwGsA\n1r2PnUovATiyzXcfds494Zx7oma8IwICAgIC3ljs6oXuve947+8HcBTAOwDcvdsbeO8f8d4/4L1/\noFAY5CQWEBAQEPBG4Jr80L336865rwJ4F4BR51yKpfSjAC7v/O3BkKjDnuTvrHY1vKpsEvUlPrFZ\nQ05JLcxTp07EbdMzpDBcWVByZ2ODc4AkxaRjVPaO1AS0xGN/bk7xZY/YN7jZVBWryTUJO5Gq6ilO\n12nNGV2OzHSs2vcQq6w6WjPM5CSZM/L57X8Qe6u603frRq18+UUqFtIyNULlHvOXSdGanNLCCKNj\nRHA1jL+umHUsUSrk4sYmkZZpU6dSiKTNsvpiFwriq6zXkMIdcl1r/cpyBGjNpP2tc2rT/Kj62ec5\nH41EipZMAQgxOyRzqjavrivJuhXVCqfsLelYShyZHDV0PkTLlpw1VRNrICSxHYvjdbaEvhBng5wD\nxKJkzTbduFiCDbAQM534X5s9LBHTJs1zpy3mKXMzT2tb4DlKmxS8smWM5RGdtph3tje5VBom106K\n5jLT1H7k+Ks2DXJC6viy84FJdYI650mxvgGSftabWp4y551Ivqx7bXllk8/Rd4s4DAixbiExHZF1\nXIjTdWtbkUnfjq05yy+ONjsn9MQOyPvGREDL9bQUybXjqhK6c27SOTfKn/MA3g/gBQBfBfDjfNpD\nAL54A/0ICAgICLhB7EZCnwHwGUfiZALA73vvv+Scex7A55xz/wHA0wA+fT0dyLJkZSMY5XOxlDdt\nnHuDpXFL+ImEPnZIf9s6TFzY6yZY0hmUQ6Ub510wjW57N0GJYLScqBRqsJGocty62El17/jyVsgR\n6cJIHI4l11Jpewm9ZQiajOT2MBJBgqWFkqlCX+XCDBFLW4tzWkBjfp4UrpRxLyxxsYS2kULk89ET\nVLSh1lANQIjPhPHXWuDcMEMm82GXpcM6Z08cKtkCIUzE5nUvDLP2kLCV6Xk9MixZLixroYi8FJ6o\naD+Kw73ZKi3yBbpX20jSSSHUzVg0clZEWL2GaBtp49ZXY4m1sqkai7iCQtwWzZq5ARJ6pyN72DgF\nRL0ucynjMOBYzO9xiBMC0ZCcSceFHCLOQlk3WgH3LWUeDtlPgxwXBCMm86YQiatl1bSmh2gPTE6N\nx22VOu2FNdbaWmZSy5skVdvnt8vZG3NG+yqy++EwP3PelEDcrJFm1o5UGhd33VRP5GevC6Et5CHe\niO3I5H2qUz9sZlF95hP8f72meKJ6s4UbrX7HhmvFbrxcngXw1gHtZ0H29ICAgICAWwAhUjQgICDg\ngGDPk3MdYrXMJtsa4QjGsTElvcQ+IYSmjcAT39z5eTUZSIRXwxAzooqKamoT/EuyKJtAxyX6CR/x\nJZX+5o3aP8pmoBGT2KjE6ruQdgDgvPSD+2+1u/j+lihlAsoUsUhsyeRj0wmn4qRRhihiYqZmkm2V\nclw3kdOprm9q8qoWV5y3vrPCw2WM+Shipkx8x9eNOUHMXbW28UeO/YUHzG1CCi+onCEmuenDWhBj\niKMay1UlNn271wc7ZUxtg9IxR2wumVSrgI6Tp95KOy2OHo1aOpYWE+NCbCYMUdlmU5FNvdzmxFO2\ncEuLTVTxvjOJpBLdfv/2ZLJ/3ryYwHgjWVNUItVvGhGyOmP2ZDYrJkS6RmR8w6M2R29GatZzKU4E\ntkNGqXJNzW+SrMzbYjGO69am1bQlhV3qTSZ4e6pksAnFzHOKk4llzJyKD31XCM229lueK5vorsWf\nJTIXMCmLed7sOIXkbJjiJeUym3h7nnM5LkS2eZYk75+N4OX51Xjma0eQ0AMCAgIOCNygCMnvFWZn\nZ/3DDz980+4XEBAQcBDwyU9+8knv/QNXOy9I6AEBAQEHBOGFHhAQEHBAEF7oAQEBAQcE4YUeEBAQ\ncEBwU0lR59wSgCqA5aude4tjAvt7DPu9/8D+H8N+7z+w/8ewn/p/m/d+8mon3dQXOgA4557YDVt7\nK2O/j2G/9x/Y/2PY7/0H9v8Y9nv/ByGYXAICAgIOCMILPSAgIOCAYC9e6I/swT3faOz3Mez3/gP7\nfwz7vf/A/h/Dfu9/H266DT0gICAg4HuDYHIJCAgIOCC4qS9059wHnHMvOededc594mbe+3rgnDvm\nnPuqc+5559x3nXM/x+1jzrmvOOde4b8DcvbdOuAi3087577E/z7pnPsWr8PnnXOZq11jL+GcG3XO\nfcE596Jz7gXn3Lv24Rr8K95Dzznnfs85l7uV18E599vOuUXn3HOmbeCcO8J/4XE865x72971XLHN\nGP4j76NnnXN/LNXY+Ngv8hhecs79w73p9Y3hpr3QueLRbwL4IIB7APyEc+6em3X/60QE4Be89/cA\neBDAz3KfPwHgMe/9aQCP8b9vZfwcqGyg4FcB/Lr3/g4AawA+vie92j1+A8Cfe+/vBvAW0Fj2zRo4\n544A+JcAHvDevxlU5PJjuLXX4XcAfGBL23Zz/kEAp/m/hwH81k3q49XwO+gfw1cAvNl7fx+AlwH8\nIgDwc/0xAPfyd/6rk6K/+wg3U0J/B4BXvfdnvfctAJ8D8OGbeP9rhvd+znv/FH8ug14kR0D9/gyf\n9hkAH9mbHl4dzrmjAH4UwKf43w7AewB8gU+51fs/AuDvg0sceu9b3vt17KM1YKQA5J1zKQAFAHO4\nhdfBe/81AKtbmreb8w8D+F1P+CaogPzMzenp9hg0Bu/9l7mwPQB8E1TgHqAxfM573/TenwPwKvZh\nRbab+UI/AuCi+fclbtsXcM6dAJXi+xaAae+9VNOYBzC9R93aDf4zgH8DrZgxDmDdbOpbfR1OAlgC\n8D/YbPQp51wR+2gNvPeXAfwnABdAL/INAE9if60DsP2c79dn+58C+DP+vF/H0INAiu4CzrkSgD8E\n8PPe+017zJOb0C3pKuSc+xCARe/9k3vdlxtACsDbAPyW9/6toNQRPeaVW3kNAIBtzR8G/TjNAiii\n3xSwr3Crz/nV4Jz7JZBJ9bN73Zc3EjfzhX4ZwDHz76PcdkvDOZcGvcw/673/I25eEJWS/y5u9/09\nxvcD+DHn3HmQies9IHv0KKv+wK2/DpcAXPLef4v//QXQC36/rAEAvA/AOe/9kve+DeCPQGuzn9YB\n2H7O99Wz7Zz7GQAfAvCTXv2299UYtsPNfKE/DuA0M/sZEAHx6E28/zWD7c2fBvCC9/7XzKFHATzE\nnx8C8MWb3bfdwHv/i977o977E6D5/j/e+58E8FUAP86n3bL9BwDv/TyAi865u7jpvQCexz5ZA8YF\nAA865wq8p2QM+2YdGNvN+aMAfpq9XR4EsGFMM7cUnHMfAJkgf8x7XzOHHgXwMedc1jl3EkTwfnsv\n+nhD8N7ftP8A/AiIWX4NwC/dzHtfZ39/AKRWPgvgGf7vR0B26McAvALgLwGM7XVfdzGWHwLwJf58\nCrRZXwXwBwCye92/q/T9fgBP8Dr8CYBD+20NAHwSwIsAngPwPwFkb+V1APB7IHt/G6QlfXy7OQeV\nUv5Nfq6/A/LmuVXH8CrIVi7P838z5/8Sj+ElAB/c6/5fz38hUjQgICDggCCQogEBAQEHBOGFHhAQ\nEHBAEF7oAQEBAQcE4YUeEBAQcEAQXugBAQEBBwThhR4QEBBwQBBe6AEBAQEHBOGFHhAQEHBA8P8B\nyomRwNMo36EAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "9KD9eaG9VYf_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 2. Define a Convolution Neural Network\n",
        "----\n",
        "\n",
        "Copy the neural network from the Neural Networks section before and modify it to\n",
        "take 3-channel images (instead of 1-channel images as it was defined).\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "6vx9dkmKVYgA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "\n",
        "\n",
        "class Net(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(Net, self).__init__()\n",
        "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
        "        self.pool = nn.MaxPool2d(2, 2)\n",
        "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
        "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
        "        self.fc2 = nn.Linear(120, 84)\n",
        "        self.fc3 = nn.Linear(84, 10)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.pool(F.relu(self.conv1(x)))\n",
        "        x = self.pool(F.relu(self.conv2(x)))\n",
        "        x = x.view(-1, 16 * 5 * 5)\n",
        "        x = F.relu(self.fc1(x))\n",
        "        x = F.relu(self.fc2(x))\n",
        "        x = self.fc3(x)\n",
        "        return x\n",
        "\n",
        "\n",
        "net = Net()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "qeOPKLaYVYgG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 3. Define a Loss function and optimizer\n",
        "----\n",
        "\n",
        "Let's use a Classification Cross-Entropy loss and SGD with momentum.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "vBPoKmoEVYgM",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.optim as optim\n",
        "\n",
        "criterion = nn.CrossEntropyLoss()\n",
        "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "YyLCAmHIVYgQ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 4. Train the network\n",
        "----\n",
        "\n",
        "This is when things start to get interesting.\n",
        "We simply have to loop over our data iterator, and feed the inputs to the\n",
        "network and optimize.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "J-Tiqpi0VYgQ",
        "colab_type": "code",
        "outputId": "bce530af-effe-4b0e-f44f-f82f1e585b35",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "cell_type": "code",
      "source": [
        "for epoch in range(2):  # loop over the dataset multiple times\n",
        "\n",
        "    running_loss = 0.0\n",
        "    for i, data in enumerate(trainloader, 0):\n",
        "        # get the inputs\n",
        "        inputs, labels = data\n",
        "\n",
        "        # zero the parameter gradients\n",
        "        optimizer.zero_grad()\n",
        "\n",
        "        # forward + backward + optimize\n",
        "        outputs = net(inputs)\n",
        "        loss = criterion(outputs, labels)\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "\n",
        "        # print statistics\n",
        "        running_loss += loss.item()\n",
        "        if i % 2000 == 1999:    # print every 2000 mini-batches\n",
        "            print('[%d, %5d] loss: %.3f' %\n",
        "                  (epoch + 1, i + 1, running_loss / 2000))\n",
        "            running_loss = 0.0\n",
        "\n",
        "print('Finished Training')"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1,  2000] loss: 2.148\n",
            "[1,  4000] loss: 1.803\n",
            "[1,  6000] loss: 1.658\n",
            "[1,  8000] loss: 1.551\n",
            "[1, 10000] loss: 1.493\n",
            "[1, 12000] loss: 1.458\n",
            "[2,  2000] loss: 1.383\n",
            "[2,  4000] loss: 1.367\n",
            "[2,  6000] loss: 1.341\n",
            "[2,  8000] loss: 1.314\n",
            "[2, 10000] loss: 1.301\n",
            "[2, 12000] loss: 1.264\n",
            "Finished Training\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "E5KoH1o8VYgT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 5. Test the network on the test data\n",
        "----\n",
        "\n",
        "We have trained the network for 2 passes over the training dataset.\n",
        "But we need to check if the network has learnt anything at all.\n",
        "\n",
        "We will check this by predicting the class label that the neural network\n",
        "outputs, and checking it against the ground-truth. If the prediction is\n",
        "correct, we add the sample to the list of correct predictions.\n",
        "\n",
        "Okay, first step. Let us display an image from the test set to get familiar.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "2Bv1HtenVYgV",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156
        },
        "outputId": "b7de16d6-3a04-479e-dfba-910608243058"
      },
      "cell_type": "code",
      "source": [
        "dataiter = iter(testloader)\n",
        "images, labels = dataiter.next()\n",
        "\n",
        "# print images\n",
        "imshow(torchvision.utils.make_grid(images))\n",
        "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "GroundTruth:    cat  ship  ship plane\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztfWmQJVl13ncz8+2vXu1dXdXd08t0\n9+wwA8MAEkIIJHtAEihsAiMrpLGNYyIcIiw5FGEj64dMhH9IYYdkOULGMSEQSFYIYUACIywDA2KR\nNDA9K8z09DK9Vnd1Vdde9faXef3jnJvnvFp6qruarq7ifhEdlX0zX+a9N29mnnO+sxhrLTw8PDw8\ntj+Cre6Ah4eHh8fNgX+he3h4eOwQ+Be6h4eHxw6Bf6F7eHh47BD4F7qHh4fHDoF/oXt4eHjsEPgX\nuoeHh8cOwaZe6MaYR40xJ4wxp40xH7lZnfLw8PDwuH6YGw0sMsaEAE4C+BkA4wCeBvCL1tqXb173\nPDw8PDw2imgTv30EwGlr7RkAMMZ8GsD7AKz7Qi8Wi7avr28Tl/Tw8PD40cPExMS0tXb4tY7bzAt9\nD4CL6v/jAN58rR/09fXh8ccf38QlPTw8PH708NGPfvT8Ro77oZOixpjHjTHHjDHHarXaD/tyHh4e\nHj+y2MwL/RKAfer/e7mtC9baJ6y1D1trHy4Wi5u4nIeHh4fHtbCZF/rTAI4YYw4aY7IAPgjgizen\nWx4eHh4e14sbtqFbazvGmA8D+H8AQgCfsNa+dL3n2b/wBQCAsUnals1Qt0wg35tWqwkA6MRtOiab\nTffFCf3WJuKxY4IYABCEqs/tEu0D7ctkG+m+EO6aco446QAA2h3pW5IYvkDE/THpvibvkxYg4XEZ\nI62tFo0hjqNVYw+4b61E2qrUDdRacdpWuvcxaHz4wx9Otzudzqpr3gxc9/nsir+6KdBt1Bq4Ru14\nZdz8Jep4N89ykmt5a63Vb3f8xz72sVX79v8kz23cSdtmrl4BADQbsmYO3XkYANDXWwEAZELpTzZD\nCy+r23g9R0atsU4dAFAuZfgc0teIt0O1iOfmZgEAPT09aVsmk+Hz0nEmkHN0khYAIFhDdAuMNNaq\nZA6NIlqT+Xw+3ddq0Tk6/AwCQCFf4GtJ3/7g93636/x79+1Kt8tDR+l3oTy3lZ4yAGCpKeu6ujjD\n/aX7najFEPEgClEubcuH/ApTz236AHJTnMj5XVui2tw13Njp+jyXa6wdw/fPBPq9EK9xHP02l6P+\nZgPpNyxtm6zMX23mOADgG0/9YNW5NorNkKKw1n4ZwJc3cw4PDw8Pj5uDTb3QbwZaLGVZW5dGlk5z\nKKVNAehLFkUseWuJg7+6JiONTSdVJPIFjFgCDLkpUucwCUnN6IgU4qTlRJ2jZUhyiUP6wrb0vjjg\nc8nX2rCUn1d9i1gyCiLqeNxuq450eEhyDieRhuH6FrIwDNfdd7NwoxK/no9UjlJSZOJEKstjsLLP\naUwGIg3JWTYvoa+FcpHubWDl8WhWqS1pCbGfz9J5SwU6LlKXcWsnpxZZIcv3XY2lGbvjaF1l1Tpx\nUxRFcm+d5B8oKd/NTY61Vr1MqrU2X1PgtFsLOW/AF8uwlOqkfgBoN5s8PjUWljpxjTWRWJHyO2E/\nnSsjz3QckoQeZJSEXl+mvsVV7oecr2npuLaSjBs8v0poR6tNWlTAz0S9Ju8W95zo8TmNOQjkObRO\ns+HJ1BaBTifmY+Saxrj3k6yZ/n4ac67Qw+eXe5a4dZ2TfsTLZWwWPvTfw8PDY4fAv9A9PDw8dgi2\n3ORi2SQBK6YOy2SUiUUlTNqkAoUFNmsotdVZGzQxkWWVqmNFpUnaYddxTnUCAGNXEHMADBM4NhTV\nsR6TbndlhtSzakvUqOVlagutnLcnz+SYIvUqRSKUCjkaZxK00n1Bal6RsbsRtJP1zQTahPDDqhO7\nkfN2mTfc8V26qdulTUQ05802zUek9eyYfhuata6drNG2MVxrLBGbvQJl9sqGdK1MIG25gM1pbp8i\nNJt1Ms2EoSLwIrrv7aYQqwHYxNahNmvkkYzZtJTNFOR4Nw9qjTlyOGazoY73mLl6FQAwMtQvx7N5\nJczKtUK+lptnZflBxMc3FUnsCNt2W9pWIrCyL+b+xuo5iA2NOd8j/RjcP0K/XZgDAJRry+m+VoPe\nEXFZnseklyLPe7Iy9+66AdtlW015vpwDRT4v9yWdUrUm3Dp2fwNl4+3wmBO9/Pjy2UjWbqHAxDGc\n2VBMOokz52qZ+iY4MXgJ3cPDw2OHYMsl9ChmyTyUr2PAkkYuVF9/xzjxlzLQzA//tKMlWEfyZEW6\n2X3gLgDA4vw0AGB6RiSZTETSeAD5crc6ND11KwFRx8+TxGNzgwCAdigkT4slh+WF2bTt0iRLGnkl\neU3MAwDu2E3XHOzRUpxzZZSxO+Ejtqtdoxy0ZHwz3BVvipSf9ltpD+za2VHiTZs1pVNnzgAARnaL\nu1vC5PbwgEiYeSaSkk308VpzlGUpPOmIZBeydJVRhFyG24KY1lE2o6S+kF1jlfaVCejeJkZpZAm7\n4zaYHFXrqcFjLxZlDYeOKdXiIc9DlV0qn3nm2XRXmzWF/sqb0rZcjp0D1BSkrrOsvQbKXdBY5xwg\na9ImjhhcX0LvQFwrA9BaT0JFCLOWFiptrcTsZqXI9/jZp9N9rWmS1kfvv0v6dpWeuaaReSvzwJbq\nRKzm1VhyrLEHg0JABkyK6ldKs0jnjdqsubRlspZKdF9yCwtpW7TvXgBAra83bUtY64r5nuUTIVZT\ni0AsbWG8efnaS+geHh4eOwT+he7h4eGxQ7DlJhenl5tI0uo6dbijIyiZgGqxGpxVZFMcO/VPmST4\nHNqv980//TMAgGf+/h8AAJfZ9AIA1Y6L/BRV7Pz4FADg7LikqMn1jwIA9o4cpGvmRK1ssbqYKUuW\ny06D1MSZqctpW7GfzDXjyxR92FDq80gPqYTFjKihcZvUZh0Mt5IOXIsUvRWRotc2zTD5llFRvexj\nXl8WEnx+gVTjyWkyVRV6RH0e5IhIHdXoSEAdPbpGZ1f0YuPIsnnPqnNk3OTH0u8Qjryntozy6247\ndTuRc4QVmgdjVdwB+zsnLho5lnW9vEimuXJRSMCA51tHbUYcWT3PZOjsopgSC+yn3VKWkVabrhVl\n9ZqhtpgjsTvK3OSitLPKx9rymk3i9c2AeuadCTFQY487PFZl6zBsEmkYuu+ZRNaCGSJTXG1J+tY+\ne5L6a8QslfB0VZ1/u3q+sm2OH7moSHmeD+1o0WDzadjguZJLormb+li/IqbVHkPPvOkdkvHxdduB\nI5pV7AXPd6hI9ijYvJnTS+geHh4eOwRbLqE3A/oSL9RUBBlLN/1lESsqTDJFLKFowip1O1IEjSNN\na7W5tO3rX6K8MZPzJHFMLsv37PwlOu78ZUnxHuZJWo/DStpWqtCXOFOkfVFeJIMcS5H5QMYy3aIo\ntdG9d6RtDSZrzpwhCX12XuWU2UPnPTAsmkKGXfeMchsT+YzHq77+Nrk+mTQNzFxDQNBSebCGhB6z\nFJawNKKjWV0E3tWZxbRtsUpjrev8HTUaTZAj8rlal3tbLrJEqvrm5P2NKiDXq6nkjHOxk/l2ZOia\nLocJRyYql8OINcpIMY+hofmwsb57PD52BIiVa9vyEs3bBX3NyEVWizS5r0Lz5lwUX3jxxXTf6+67\nDwCQaJfKmOY3r116WVOo11gDjuT8HdYQw0icA9qcL6jZXD8ldqyk94TXsNUyJDsxtLR7I1+3d4nn\nangk3VfYtZ/6Y4WMBLte2qHdaVM9w7lZrlBeGCgX4Co/r3ZkMG3LJNSnhtLwS6wltpZofE2dY6fA\nEblVuS/RIGkPJqPcMjlfSw//NFQaQMfQ3JtAuehi89HeXkL38PDw2CHwL3QPDw+PHYItN7lcrZOa\nMdsWUvSbf/e3AIB7j4rp4qfuI7Khn/3VNRnjkvAESn2JmXxRXBrOnic/59k6qUK2OJDuC8tMvg2I\neaDA9U9bKmVqi4m4Sj/1rVKWPk5dIRPK4pwiS1glzBfENHNhjsjYTIXUyakJqS5VvrIEANhdkeML\nLlVvosi0FajWdHIzVjmVqulSC4cq0ZPbdulAVU4sBMnqb72LYtW2jmU2BzhytKCIswZH1E0ok8vU\nHG0nijBrsz2ltkQE8tS0zN/4pQkAwL1HDqVtdx7YS/1XfvkpOesifbWVxXVbhylcgyoN2eSXtMWc\nELCJr74gYwGbGywndQoLMvYs36usmm/TJlNbrM0UHA1tUiJWzE3VKpkWJifl+FKlzNdUicl4zlvL\ndFxe+cNfnSdi9dkfiBmmlKNrHj4kcxqx6adZo/VXiFQiqSatrVilkY7do9ZQ87ESaopdCtukK1aE\n96lnOcPmrtzpU3T6Z76d7uu8iU1VKg2t5RiR7JI8Gw3QPJQ53iPMyfFJic5vrCLqOTlez6C8gzKX\n2FyzTGsyMyLOD7hI+6KKmEUbV2l+w6K0JUfJN73Bib0CReJnOzQ5kbIl2mtw/BuFl9A9PDw8dghe\nU0I3xnwCwM8BmLLW3s9tAwD+AsABAOcAfMBaO7feOa7ZgV6SEmoz8m1pZ4l4nK2p5O8tciOqZNnN\nSxEpTiINQyFtGi2ScK8q/ml6ib7OxT4iRPqHhaisJiRpDEFF5TGB0sqI1NSokgTTWKbj9ytypcbS\n+FRLpGXD0tLCrJLKWFqp89c/zEq/JxdpGicWRCvYP8QayDW+4PN1GWi5SFpDoPJKuGIdXYK3I2tc\nEG5X2to1vvVruENemSCXzoEB0nYKeZF8mg0aczEnbbuHSdOySnyr1misJZZkWg2V7pQHvdyU8XXS\nPBvKjS51n3T7Vg2zS2K8lrdl3hUwUAc5CT2ntIIyk8+9TGYF7H4JADm+x3ktkLIWFTRkLaRFD7hQ\nSmtR1lpPifb1D4gmeXactMAzF6+kbSdPPwkAmJsmiXS5IeeotanmTATlhsiS/wN3HU3b3vuzjwIA\n9vB6buZlnI1qlX8n16xwAXpTX8J6yISy/lz6a0eOApJCNlJyZXmOrtUZJzffitI2li7T9Vt5ica0\noPeCuTKVtpXGmNCssOYJeZYK7C6bnZd+N5iI7kxPpG1ZnsPOIs1VblYcI9p11qYKouHMnyVnimxB\nJPSeUSJxXSooq1wUm44MV2u4lWxeRN+IhP5JAI+uaPsIgCettUcAPMn/9/Dw8PDYQrymhG6t/ZYx\n5sCK5vcBeAdvfwrA3wL4DzfSgbte9wgAYPypE2lbuZe+/o+89c1pWzEkO3OLJWQtfRrORhdbyffR\ns4vqVz//4ik5bx9Jh3v2kyuXVba4DEvhSXMmbWu1klXXCvmL+tILLwAAKipBfbFEkkFJ2dEuX5kE\n0J1nJmSpY4DdzebnxP43N0vbZyfENWtshFyyoqyKbliBqCKaQszSdVvX32PbZPoXYtd0wSpaIrVr\n+DA6AV55SKYBLi7fB5TraB+7frXb6lwstRXLYpN0ErrhYDGjXMRyBefepcqqMTHSZXNc1Te5Zqb7\nEN69voh+8dw57rfM99Iirbu4LZrCpUuknczxGqguiz151yBJ1eWSBAWFXJylpTIURpxrKOBcQlUl\nvTfcYFShjQuXiX85Oy48Q7VFv833sutcSSbGrcRSVmS3ifMUjHP58mTa9u1v/x0A4B7mKob7RCKt\nL5Pk78rDAUD7HsqnsrywvmKey8rYrZPWE6Uys4YTKDfbZQ4EXH749QCASvTGdF9tie5BW+V9Mjme\nG1WeMVOg61bZPVO727Y5X0pGPRt1nhvtNFhnu35tma5ZKshYGnx8rizP+UAPvXti9a5Y5rULdqMs\ntFXGRu6T9jBu34T8STdqQx+x1jr95AqAkWsd7OHh4eHxw8emSVFLxst1Py3GmMeNMceMMcd0nmYP\nDw8Pj5uLG3VbnDTGjFprJ4wxowCm1jvQWvsEgCcAYGxsbNWLv9hLpoL9h4SgqbMF4o6Dh9O2IVbb\n58+eAwC0dXRZh0wXj7z9F9K2Ow49DAA4+MC5tO2Z58hM0l8mE8blKcnlErEbU04XV+DeLleF7Jqf\nJbVzoJzRh1A/2KwyNCy5XFzRhuk5MaEYjqbsYZfHKFTECKvcr14cT9uG+0ktP7JXuU6twCf+5H/J\n+bkfGaX+lXtIZTx8UIjgN72O3Kpc2UurzEKOZLTavuJy7CiziiPssjk6vyY7s1kyoQz2K/dJVxtW\n1WhMc4Rk6ByNjpx/nknieZWqdGmBTABt7arJROYgu54dOSyEVcZFE+rC8EGXAaYL3/77p3i4qsCK\nI7LrshbOXSHiLq39qcSj/l4yWZQUSZzj4zLKlTFil7qAa4rWFKEZ8Tmsylt0ZZaI9LZit4s9zt2O\n8x0tK3dLvh+NhvS70kPnfcsbH0jbqpzyucEuuhcuiCnl1VdfpbErF7vzMzT39ZqcN8oJuQ8ApZI4\nGHR4HtqxvmdcaEaRgYZNUIURIj4XqzKWqws0dqPccVtcMzWrycV5+o3LBZXLynOwyGs8n1GvPpfW\nWEWKNjl6GVwzeKEua9Kl0SmqaNqevWTiDbUZMK2Hy/dK17Jwbw61KJOb4Ld4oxL6FwE8xtuPAfjC\npnvi4eHh4bEpbMRt8c9BBOiQMWYcwG8D+B0AnzHGfAjAeQAfuNEOhDkiFi5PHk/bHnwjJeMv9coX\nP1wiAipmKSFS5bPOXCTi4m39B+XERQo+6SmpKu0RXavAboL5rCoVzl/nPWOjadPLLJlkFbmzyMTM\nwX2kURy9+9503+wsF7OoSIDCZXanMoqE6esnqXaBpU+d/6RQpN/Wl6Tfpy5wsIcitkYkdQUdX1PB\nT3XazqggnyUWcIuqLb7nbgBAwzJ5pCT0HEtKWqp1hSp0FsLeAdJGUuJJuTs6N6xQSeMu0kvLIglL\nK+c48OvSlCh8szOkEdXrItnFTZZEVc4Xl1Nk7z6ic+7YtzfdV0rXiiZ915fQnz9F/SgWRCOyrBE2\nO3JfejlrpiP/WkoKvrpM9yBUc9WTJ42sEwsJbpgEDNm3zUQSqJarkmTZagvZOjvryFBdLo3+tjhH\nzFJV5qrF7qz7hsX1cbCfFo8LXAKA2TnKAzPYR/14+PX3pfvG2TV1oS5r+JVxui+BWtcHVzBpkcp0\nWuihZ25ZlZSLWKWJVZbBiINvAl6TiXK3NFzwJlLXdFvtlsowyVp2xJK31ogcGRorLdCVtuuoVZkp\nMGkZr87a6nK/ZDpKU2CPAZ2xMR+7DJ18LbXkXGBdtxfx5rOjbsTL5RfX2fWuTV/dw8PDw+OmwUeK\nenh4eOwQbHkul0yeCJpGQ6vPXL9RRVAWS45kIlOArjdajkhl+uQTH0/bfv6ffZjOoaLbslxL0RXL\nOHhoT7pvapYIrsayqM27d5Hfui4Y0OQ6j4cOE2F752Ehcxeeo1qO1SVRKx2p01ERcnU2ifRx/cHY\nStRabz+pix1VkSAMaHzjl8UUMfI6dOED/+SfSh+ZLCyp/DGOhCkoU5VLLbG4yPlVOmIKyDBJFyn/\nW8uqa135Z9uEzueqomsiNuLjMxkdgbrabOP8bxuc/6SkcmT0cz6duCV9y4c0rvkZMRmMXzoHADjM\nRHoYKNOSdRXtVYrha7j8LrJZz2rikWMLCqHMx959d1L/XZrgK7LWptlUNDIi9VFzQ2QGqs6LP3fC\nkbC9/WSvyOUklqLBQ651xOSS5+cgbssaC5lcdEVfMllVaCNP24+8QUwoR/eP0flbstbPvkrjevXE\nywCAt75JCNN9++j4Cy9KzqF27HIqrV9TNKv6keWauokVM2eBSfCOSlO8xJGyMROf+V4xFY2U2ASm\nyEO3rrW5IoSrmUp/dWGOtWD52dQml5h93V2a4kBdM+sMPSpRVJPfKTp3VMQmxxicP0YXXeHnRtd1\n1abXG4WX0D08PDx2CLZcQjccQVZTknGDJcyMzuMwwy5FnK8lg/l032gffTFPHZeo0Mvjp2mjJqXf\nzo+fAwA8tJuiU/fsF2ZxbIokpOppkUIGciQd9vRJWalXXz1L1xwj6X5+UaSnNn/pJ68qCcyRJco1\nscYSuuHcDpoKKbnsjYlEfmYNzUdr+grWQ9IWCSKVUNT+cpbOW8jLnNY5U16tTf04d+acXJNJ0TsO\n7k/bzl6kufzS3zyZtrU5w2We87UU1flddF1vRaIO+3pJynroIVExhodIKr1zL81poNwFnZTliCtA\nyK76LpHexkbpXo3tIVJbZ/CrsWtbl8ZyDVEmw0T98K6xtC3PhPT0tLiTVjlq2YX7NVQEaO8wra09\nyvW2p5fGWRkSqX2GifSYJba2qujmXCRrikhstR3hKRpL1mX0zNE9zljRoHbx3A/3yz3IM8E33C8s\nZoVd+2YuXAAAnH/1XLpv9wCt/4XJp9K2DJPhrXD9V0ikcpeEnEUyr/K7zE8RwTu7LDlUrk7Q/Pb3\n0Pq//17RFDKsnTcVIdxmDUET+m79u6IvgSLqnZSsSyfGKRGrWcvu3EA6kyvSc8gzF/Hxeu2632Sc\n5qQfdD59oFww42u40m4UXkL38PDw2CHwL3QPDw+PHYItN7mkqW+V+jI6ROqWVt+//iL5hPdzkv0j\nA6IC5XNMCkXii3116hydvikRb3fcSX7qIZ+3WBECamiECKuZWVFvF5gM1YXNd+0idTlic1BDkZcu\n6VJdmQc6/OOOOkmjyak5O/Q9HVQquOFag1kjY8kxaRTb7kg8jb/6P19JtxNO2B8oH94yE8w9yvxx\n4AiNeXiQTAyDoxJFOsB9yqvkUvPHyRz1/eNSd7VuXTEN+n+k1OEK//bwHWK2eesjb6BrlcTHu8Rq\nu9N4W2pOO+xbXVsQE1ub/bgLRelbXx+ZGyY5Gdq0KpJR4IjFkd0yz8WiikFYgX42sYXKnNDkQh5G\nyUCzM9SnxUVOg6xMhCFHGJ6/JAmwKotkLuntlTgF53/eZKcAowjCnItmLMl9L1gXWapzAdMzUSqw\nOdKKOWbvIM1LURGU1UXqd0eZclzxj4NsIjr+ypl039GjlIgLigC9fJl80/P9YvYC9HY3CeiKrSTK\n/LHEMR1Xr4opcX6Oznvyxe8BAF554R/SfYcPU8zHgcP3pG39Q2w2UuYKlyraFTvRhoww9WFXfUsL\nvUibq5ErhXQU6crHa149jaxeg21PSdeu5Hd8VnW/9bvkRuEldA8PD48dgi2X0F0UV29ZCKu+Hto2\nKmfIoiVJY3qOvpRDPdL1EhM6cSCSybnL5wAAI/2SDH8/f+GdO9j3npHo1EsTJMn3lEVqz7Bb1Uun\nL6geu0hH+ttUX9VljtDrUwUJOix2TkyqBPw91KeIXaOKRZHAXP4TtIVYjavUt5Fd6+dyefq5H6Tb\nhQwRlM2mELZZJvXe/JY3pW3nL5GkPcOc1P33iWtblgnNWlOk/AxrNm94gxCaDY5EzLI0eeSQROve\nxylWx4ZEIq0U6d4myk314hWKUpya4+Ie01fTfVUmy+fnRUJvcQrbjHLBdLlkXCRxWxGUxT6at/sh\n4+vtXX8unaRdU5GooXEl/EQriDkVa8QRyIkV+Sibo/MPDUnkcZnXeF65gvZyvyO+Z9qd07JrYEe5\nk/ayS2egoisTThMbuejKpkjevZxAxnZEa4xZ62mpSMc6348ir83zV2T9vfwqaX/NpkSgths0vzbU\n1Pv6cFJtPi9jv/suilQ+fI+4D9eWSFp/6VlyAX7umBCx3/4WaYjHX5a1fvSeBwEAR+4Sqb2vn9ab\nI4vDrj66+V0j97ImW13JvM7qso8uejRWJGqSuk+uj6701MaVzZQ1rFNs3yi8hO7h4eGxQ+Bf6B4e\nHh47BFtucnHRe7t3iU+4qzGYKHJxdC+p8sfYlDJvJEWtDUkt7x0S4rG3wj6geVGtD7DJpcwpe//4\nE3+a7qvxtRbrQqbV2A9YZ9rczZGcjVlS/6o5fU0yC71yQvzhJyfJfLCookf7+uiElRKpz6EisTIc\nvRfWLqVtwyXa35sXhU4lIQUAXL2o/OcHyGy0d6+QgPe+7gidPyfneOl5Ip5GWA0uq2pGU1xfsVQR\nk9VghY5776NvT9sCduju7aXjhgbFf36WUw2fPS/zsTBPZqDFBYmOXWLyeZ7TFM8uSgRohwnejEpr\nnOUKQYGKrOut0Lj6OLK0X5mncmzSyhbEtLVcF9J5JQbZh1z79pe5+kyi0r9mApqPXeyvblSUbJZ9\npp0pCADyHC0Zqjy7zsSSVmlSJhfng1+rytpxEYs5tSgtm19qCzTfl87JfM+y83NfQY4f4RTD+byu\nwcsmlIjMTVFRyPOrXN9z36g8cz1czWuxuT6Rl6i0uC6Jlw10G/UtVL7pfYOUhvZt76C1e/iwmPC+\n882/BQCcPSvPRvU5fm4XxST3wOuo2tG+fXQunZ467tAaj1XfEjbtdlXpSuvnur+yy9Xb1QS5s5Zo\nn3dHkKbX6iJF+R2nzDbahHOj8BK6h4eHxw7BlkvojgSs9IuE3ompW7lI3MCOcmGGY8+Q5LWYkQi8\nxJC0N7JHvvQvHyd3px/7yX+Ztv0DFy6oVklKbLekwMXUFeeKJ9+4Za4BGKmovP6AJPg9BTrHwlWR\nhjohScYju4RYjdnVq64kwkadJNIqk2+dRCSwdoMi5XZlRBIcK5Mk1exI20oJ/dLJl9LtRSbOfv4f\n/Zu07dFHKTnm174u7o27mCzcVeQoUuUKl+fouZFekdR6eDuv3AU7LNU4SVTnrLlygiSpC1Piutfi\nQiVRXtLE9vQQibyLJcZ2azURlVFFClzOC537oqeHxlKp9PA+VaeS8+lMTsr9bjTWr55VZOm0rYjb\nArtg9lVE60nSVM5EaBZUndSU9FLSYWK5TctRrriI+6vIug7f704sfV2coTHoBzfDEvryAmmDE5cl\nOnpkgMbSV5Jo5xpL14nSFDp8RkfE7uGCDQBwF9cZffBeKRpy8gw9L899XxwLVkKnjA64AEUQidad\nYaeAWEVXuvSzAZPER44KAZ+wm+/ExOfStrlpGuuppmh1k5eoPvGdR4h0vec+OceuESKpI/Vu6bS5\n+IZKqRtzjVx3H9csiNKVU2b1/jRFM8+DPkVaTEaJ/l3RqDcIL6F7eHh47BBspMDFPgB/AioEbQE8\nYa39A2PMAIC/AHAAwDkAH7Av2Ih9AAAgAElEQVTWrl8CfB243CX9QyJBdPhr3gikMEK+zJIGZyi8\ncFGCEd72JnJHayzLF7PYQ26CE5ck98bpk1TtvOOqgStvpirbbXsGxc1sYYEko96ySKR3HaXcEk+/\n8AoA4NnjZ6UfP/UeAN1ZIs+cJgl+XmVsdC6PjTpJ5vtHRLIrcBDJwIBIxjYiyaHTWt+tqaFKgT3w\neurjO9/1zrRtsI9s2z/+ZmX/ZsmuhzWFSlmk5pCLNriq9IDYanXRgYU5sttWWOJJVAaZQ3fdDwDY\ntVcyUs7OkWbT0yeujC5zn7GrK7I7O6wrjQYAy2xTtqpkmCuccHGCbP9OCwKANhf/0PldiqX1A4uq\nrE31qAIXLshoSuXpWeRgp4SzMh52ATgA+jj/SZjR0idtay2mxfXMasydNJrS706L5sqoghi2SceX\nlMbS10caTiFLNu7IyDrpY+2ut0fWZIvPUVPZJFuc4TTgQJd+pZkVOUvpuOJpWLjGfXcdSduuKndT\nOpfmA9hervqW5d2JfhBZcnU25pbS1vbuOwAAOHDgQNr29CTd744qj3d1ap77Q9L78eMvpvtc4NSd\nd0q/R0bIbbKnR/gicIBfo8U2d/XsZVgj00FEzm1RxxVZo10jaVTp6dOCGILwJhS42IiE3gHwG9ba\newG8BcCvGmPuBfARAE9aa48AeJL/7+Hh4eGxRXjNF7q1dsJa+yxvLwE4DmAPgPcB+BQf9ikAv7D2\nGTw8PDw8bgWuixQ1xhwA8BCA7wIYsda6vJdXQCaZ60bCNRp7B6SoQbVOak4tFhXFEWCuVuTJl5Qr\nXI1Um3JJcpFw7QGcPylq4iUmi976Vkqfq9OS9nA63IExcZO6MEtmlXpTJbcvkXpbGSbS6KEeqV15\nldXxc+efl7HUyDwxvyDX2jVMqnGvpf7sL4ur364KF4UwYkJxKVNLSoUVpz/CobsfTLc/+Cv/msYX\ni1p+4jQRk4lROXCYPG2z+jc7r5LWJC6PjdCvrrB6AiG2lhapJ+EkqcaXVT1QV6gkaQjZVGIC9swp\nMYWd5ZStzu1vYEjmw5kHFhaE9JqZJmLQKhNKwO5wJnB5TVTkMROweZ06eHklrSzIsYvkzLSM5dU5\nuqaLsgSAvn4iv0dHaem3VFRhu0Vmm8RKHxfZLFZX5qCYIzhDNmfp2pXOrJIvyVgK7K7YUGs3YSKx\nVGY3WLVOshwlqQlkRzA3FAlo+DhHSrZVEZPxGbKk1lQNUkcq7h6V9b8SoTI5pNvqmjA8X13ufO43\nZtU+F2Xa0yPmoJSs7Cpe4kx4dK2lObmPz3EK6pdeeDptGxik+7h7txDBu0cP8DXJDDOoTLHDXNDX\nKOLd3eeOMgN2mDRN3Ra16yObu6wyv9lkpYnm+rFhUtQYUwbwOQC/bq1d1PsszeCaBl5jzOPGmGPG\nmGO12vqeBR4eHh4em8OGJHRDKQA/B+DPrLWf5+ZJY8yotXbCGDMKYGqt31prnwDwBACMjY2teukv\ncSKRgspUl2aeS1S5NCZThgZIejsZSDa4qVmSfGZC+cL1lukrevf9QnScOUeSoCsioInKI0eIJDly\n8M607fwESSQvvfT9tG1mmoNUuAhCv3JVG3+JJPqJafneGSZ2QxXgNLqP3L/28xf7jh6RwPJcyqrZ\n0IEPJFFpt6qVeP8v/fN0u383SU0v/ECkYEcutZQUEDNJ50qtaVLGlfaKtQTBbUGXGMC5UzgL5vSM\nuCg6tzsVS4K+Sh/3RyTd2RnWRlhKnJ4WArTJ2klHuX3GXAYwVLlcinma55xzadQV2V3yHoj0VFBZ\nJFdinoney5fE/a/EZPXdquCCy0hZ5Pw0jbpoVXNz5N7abss4a5xrpajcPnsrtO5LOfpbUGRnxFJn\nrEjRTqfF51XZO135s7QYgyqawFpuWz15UcikXqJcaTmb5MxV0kSmZ8TF02VFnFP5dJymlesRbWol\njNUSOv3VRKFhqVbnOEklbf7rCEgAqC9TP65ckYIYly/T9kJRjsvwOnIkf0nljylGdJwmyC9xUY1T\n5+SdUq9TEZdOTOcaGpZiJw88QAGKRw6LRD88TGuh0ivOHbkCaRIWfH317HXSJI6KmL4VpKihnJIf\nB3DcWvt7atcXATzG248B+MKme+Ph4eHhccPYiIT+4wB+GcD3jTHOOPwfAfwOgM8YYz4E4DyAD/xw\nuujh4eHhsRG85gvdWvsdrJ8V8l2b7cCZ06Tm3HFE0l/mA04D2hLiKmK1SYgRIVHLXLTh7rvFD/hr\nX/kyAKC2IP7qxUEir06Pk3Vo314hUQ/eRYUXckqNP3QH7Z+fFff6l7luacKEy/ickEeLTOY2YjEf\nLc6TWWeXIlzOz1DbwD4yP8zklE90wiSqMq/YiGspJqK+r/Sifu75Y+n2i9+n766BmHJcvoxIF2FI\nU8Fm+BhR1SNOt6vTnbp8KlnV34D91ENL+ypZiZIN2CzVDpV5gCNnldswspxrpV1j/+iqmKxaTBqa\ntooeZZtPS5HmMUeDVpfo+KK6j8O91I9ImTqcZWMtanRgmNZJvyo84go0RGo+lpaJmFxepv7mcmIu\ncaSiTr86NkJkeC4v5gFHhlrOJ1JtSI8aTDjPz0l+oZlZ8vWuK/POPZymOMO+/d0FHbjeqVpPTa6F\nOp5GR4sPeYvNWbWqnH9hnkyPWRX16sb+5Ne/nra9/c0PoQuqeEPi/Ms7KkKTTTLKHR4mNQfRvlBF\nzr7w7DMAgOU58XcfZP/6ixPSVmEf+iw/N4mKsK6U2R9exQdkIy4MklNxGAGbcefIzHTurERiz8/R\nvD17TOXu4biNffskmnaMC8aMjtGzPzYi75sSp+k2BVXvNFg/NmKj8JGiHh4eHjsEW57L5fnTJC3f\ncf8jaVsC+joaTQLyF36RCZr5eSFtBgfIZe89j/5U2vbg6ymPw2c+/5dpm+G8DL1cfX3PmLhclZms\nCzsimQzspukZPShS1gIXJ3j2eZKCJ5aVu1SGCNjeUSGKhg5TW1dhBHYTPMFFO05fEQk2y+xRXUVG\nVnkaOolIFe8W4REA8O1vfjXdrnHmuWxGlS4rOlJWbnloOX+Hq5Ke0RI69SOfU4Qtu/1lVZa+qERj\nzWdpnDmVj8KlCjEqS6Qjt9uqcEaDCc9UqtURdny8Lm2XhvgqibivRNu9JRpTuSBScC5D58sYuY9G\nuR+uRJtJOu3mGLFLZdxF9Lnyezx/SjTOsxRer8o465xhsq58Tp0mFGScG5us+RPHXwYAnD93Lm1z\nUc5WuUOOjZIDwABnvKwrbzK3PT8nhOYMk751pQG7nEPOE21+UbSkgOe+GMnacflirlwRDXilhN5W\nRTUcKW86cg4Xlaqd9SyozZGoy8syWa6Yyl1HRZt/w4MPAwCeeVGKXjz1NGURnefiKHFH7sGuUSI3\n3/a2t6VtEd/nc+fFxfmppygX1P33UhR6pVecKyZ5zJOT4gDg1u7uEXFvPHjwAF2fHQuqS+L26RwM\nMpFoBY01chhdL7yE7uHh4bFD4F/oHh4eHjsEW25yOblAKv10rFKPZkgFD1pKRUlcDT76OzYqNoef\n+DEiNPMZUUMP7qfIz599/wfTts/+5V/Tta7QeScWRNlrNE4DALIQlXe2Ttunz4taCVaL7DCZdPpH\nxPyQ1hVU0ZgJmycSIyYAl4xqgSM58xmVhIxT2FaNSi7FZKRNtErWrZ6NDEv03ESdCKI4FjW7wnVO\nI9W3xWkie5cWq9wvUU0Tpy6vFb2mzCqZAt0Hm6Hru8RqABCwzaWokpW5yvRxe7U5DZwEymTFdpFn\ncrOgzB8DPaSm7lMxAHtHyf/X8Z7NhqjqgaX1FKnIvr4Krbua5NpKcfIkpYS9775707YCm1D0dARM\nPyYcHTipomRdsrdmXZk12IQYK7PKocMHAADDu6j/uvBChs08fSpRliNUdZlM50P+yglKG7usCmK4\nfTqGIWGTUnVJ5qjG/axxNGtLmcRcMY0Lk0I8uhqv8TXqYNquCFDrNlK4KE8VxIrEEal8qwqq3u5P\nvONdvEt+4IpXHH1QTLb3v5Hq5rqyq4GiiV0BlkOHJN4k4jk9cETS7I7dQURzgSOOe5XJxY3LFXAB\nxKyya1jSgLtkXyGbqgLF/sbs4NBWdrrErD+XG4WX0D08PDx2CLZcQj8xT9+UL3xHojEf3E/Syu6s\nEAZFlhJGd9MXcHRIpJY7DzG5aUWqmOC8Kp/49F+nbc88TySTi0TtCry0jpSSc8Q5ukasiT52Beww\nwdoJFGnoZlOVkmq0+LzqSxwxQRqyNGZVrpMOU0QZ9TV3pcha7fUjyWxbJPreEkkcS4pYbccktd19\nz/3ymzGSVqY4OnBKRQcuc14Xna7BSZY2lvOWIpJC7n49pSW9rErLXV0kDaDeEomxzoUldFRqjl0p\nS6yJ9KncJcNcwX10TCSfw3vIrXBXTsTUZXZ1nGW3vjAr81csEQleVhG5g5y/4/JZIcIc2izdN5ZF\nwwkcGalETFe8ImbXxFOnTqb7lhYcMS2PmCsCEinxOuGQwYAjbaFcMQdZq9Jka41TLtfrMqcXL453\nHaeCD2HZxbPWknvmpOvqtGjAGe6nK/nXUZGUVXZb7ChXSYm0XF+qrCvtJGQXzMiqCF5+XjsqgrfD\n8+DOr8vYOYG/ozQcVw6upXKojN3B+ZgSTlGbqCIS/JyfvSCuoPWWywOkCqb0Huy6/tyCXDNiibtU\nOSCDdfmQFmTMlydn+RzU8ZxKB+4CYE1Z1kdjbv2yiBuFl9A9PDw8dgj8C93Dw8Njh2DLTS7LrIZ8\n7VlRV0++StGj736jkFJ3jpFqf/YMRWq+/U1iOsizqr7UEnXuM39D6TGffVkSLNVclBqbPAKVqtSp\nRYGKbnNmklipc002hbRZJTTKt7nJEZeaDIqi1fUvi5xIKAtXgTzdhZhJRZ0Uq8MEYrZHqvyszIU2\nc1kSccVtUt3qSh2uXaTEZAOqwvowp5XNcJWcgsqiVQ9dBRZtl1qtZtfqZKZ5O1eNuu8eSV514QKZ\nM2bmJdK26cg2RaZFTHQXmMUaUgRoX6nEV5Z7cGWaxnJiWpI0GSa2KrvIjFSoCGFaZBJVp+UtK5Jr\nJQp8z1rKrOHI6q46mc7/nM0VlYpEL+fZp79cElIv5HEVVbSpM3GceoUSuy3MiilggSM6Y+Vznsly\nxKpaTznW3w3PX01Fm04xcVdrijof8hj6e2U9tdg8V2Mn+Y5K/pWk5hWd/5Xnw6wvE37rW9+QsXSo\nalApkvmIed21lVnFEfMuIZl+ltps2tLPoyMcG01pi9MKWJyKWtUPHegjc265rCtm0Rg0v2vS8bmE\nZyqik8ccKBNKxEm/ArP6ODeErvAKw++PohwfNNhcqAjv64WX0D08PDx2CLZcQh8covwWs3PyeZzg\nqLa/57qdABC39/MWfQmHd0uUpwnpC/y9YxIt9tdfp0ivZiISAfhLHQSrv2MxS45WfaadO5qWElyU\nZ4YlA6M/p5yHQpNerhalzj0T8vVDyxKHVZoCS/labB/dTdJkT0VJlbVuCX336EC6PX5hnMekiwnQ\n9tmTJ9KmBXYndFevKrfIKktDSdzFHNPxqphAq0kS3bPf+QoA4B0lGef9PM56r0jLjgTUUcANJuwW\nOHpTk7PnX6FovOm6RC42MnT9wi4Zc/9ukrhyFRpTqCJFi+z2lysKyW7C9Ze+c42NO3IPXJRx0lHa\nGo/dkaIFFUkZsNZYVzlRmrOkLV7QxSl4HlwKWZcvBxDyPJNXWgFfotWS+VuaI4m80Vjmv0JkuzuV\nV2u+XecUvKr+qyMw3V9NRjr3wo7STixLtdnM+kR9XkUqt0O+Lyoldo6dDhLl6urcNgO+piahE853\no7UCFzGbWBUFzKO2rm6nUSQ0375A1cWNQk5Z3ZTI1pQg5eHpmqVt1pi11u3WjFHPxsr3TEtFvVo+\nR0O9PnIhaVNjY/txo/ASuoeHh8cOwZZL6E6azagsgJ0GSVdnJ0Uqa1Yp2OPtb6AK8oU+yZmwwMUg\nvvldyThYZ9tvW2W7y7HbmJM+1qqgFCppIf3YKttajiU740SlQB2fIymkoMqfORentgqkWWKpzQVl\nNJUk2NvPLpujkii/zP6QdRUIsvJTfMdRyeS2yC581fFpdQRn3VPuaLN83SyPuaXs5WK3Xe2W1lWQ\ngHHqRcqfcXFJJJ/hgOajS8NhqWVZ2euvWJIKT7NNdVzlAKkVWcO5QwoMjBwkCSbfJ66r6X1gqalc\nFk2hyPb0QK0xew3b7yLnCaotidvi1GVak42G9M2Vj3N5PPQ9dppeoIKZMhz45ngVQDJcRmxz1y6K\nbbYj63wwzSatnSXlHuduW6nC7rBKMrRtmufmsqx1VyRjQUmkTjJ39mmj7OWJXR1c5nLbmGT9oiuJ\nuo/LVeJRiqG+B/Q3VovZBUC12A2301GufFzIwyppXLJaynPYYRt67LRBda9dUJUWnq2lfjYbOrdN\n3HW81txtyufEqs0FFeoiMd3XDFu635w7p18XvqHtMXgJ3cPDw+NHHv6F7uHh4bFD8JomF2NMHsC3\nQDUVIgCftdb+tjHmIIBPAxgE8AyAX7ZWhWpuECnJpInBkFTHliJtJpdJLXr2BBFL76mJCrRkyRRx\naU5MEnlWuTs1OUeDVUxXAzJSUXxuX5dbmnFuT3KcDbpTzmZy4oK2zK5eLZWC15lftNnBmViqHLFa\n7hPzSj/ngmiplJ+vsEtbRrlrvXGFVlbpF4JweITyq0wok0uq/qnfNNms4upNatfA+BoRgF17+MRt\nVtmr05LvI8hxSmLlMneZr/E8RB0/HfF8lEmNL+2TIhnDY5STZ5CLTgBAjl0BW6onls0CuYir3Eea\nmHZtirS8hm/YlXPkQqursDsV3OiIX07f66q/a3U7y+YdncfG7deEY4dNDMvLXPO1qXOusMuc0S6E\ntC6yqhjDyJ4xPgdFdC7OiZtohwtWWEVCO3NKraXNMM6c4XzssOr4jBq7KzxRqykz4ApcvChOCqcm\nqB8lVSM0YltR3FWSg+bURYMmiqjPcq4f3eZMNLFObcTz7EhLo3KkOLJV27ZcPhh9X5x7bRK7KFJF\ndrKJsitnkyvgYVdHtrpftlWeqHiA1sWeB8Q1u9fd0k2kdNmIhN4E8E5r7esBPAjgUWPMWwD8LoDf\nt9YeBjAH4EM33g0PDw8Pj81iIyXoLADnZ5XhfxbAOwG4UvOfAvCfAHzsunvgyAZdOICDXxKV98Hl\nUzk7RRLBJz7z5XTfO99BSe7PXhbpsOqCBdQ3K+My1bGUUFRuR1kuXFFfEunaERdWkZYZJiidBKiJ\nMCcJJopAqbOLmm5zx/WxVD2okuJfnaHAkvlpyfA4f56CqQ4fOoj1UMiLxJbjAJaMymcSMzmmP/6d\nVHLh8emd15ASuigyloaWeXyvKKmvl8vTvdKQQgAvsfYyUxHJdXAfjWv0IEnjfcoFM8dukIHKx9Hm\ntRJGqpQbS8RRGmQjx6fStXYpuwYpGibsuqdcR1P3Qn1e1tYC6yQ2OUeTXTA7bVlPTuLWFecdHHme\nyeoSgVw2UJPKvBbzOeX+V6DfzM7QNXUWxQxrnKGuLs/aaEdLkytIva5AGlfwQ2k9y1xEpVaVfDAr\nEVhVvtBJq7FItU4b6ApOCtlt0TrXQKVpsWSs4qzSubfKNdHdCCs+iimcFK5dizt8/bZyCkj4HWRd\niUD1PKR5mVRHDFaPxTL53eEAxorKR7T3AXLuiIzc7/mTnM9qr2ij14sN2dCNMSEXiJ4C8FUArwKY\ntxJGOA5gzzq/fdwYc8wYc2wtrxIPDw8Pj5uDDb3QrbWxtfZBAHsBPALg7o1ewFr7hLX2YWvtw0WV\n29jDw8PD4+biuvzQrbXzxphvAHgrgD5jTMRS+l4Al26kA4NcqbyhChJUOZItG4o/t0ur6XyJv/m9\nF9N9Z7m+4XxVmJHZZVKbFbeIEqvvHVa7cqp6vVPV8wWVJyJwPsKi2juf2Q6bGIz2T2UVLFYV6lvs\nJ1tQ+Ttckv2BITK1tBQh3OSCDvWcXDPh6EFdEX4l2iqis8r5OHr65JqNKqnZuoBCzOphmrFVpW41\nq60CKaxKD2yZUKqyj/C3VVGS8zVqm1H5KqIRqoA+unc4bTs4TNuDvTQvgYo2rbIq21DEVsSqv675\nmeco0Iirr+cLIjzkeO51FOa1kKyRR8Qpo1aZfiyzyalJR53DRRrG2mTA60ivO7fGHEnbZfVK3HoS\nUjlm8rmVkXtb57S2ztSSaAKUc780lHbsxmW1L7Y73pkrVD8iHottCZE9N0NmtHZr/TXZUX7oMR/X\nCjQh7PL66KIo3MTPUqDugUuRm2jTCJvFEpVu2hHSzvqhj3cmM23lSZx/uDKxOTNTaprR/uVsFoIm\nbJ3ZRr0P2pzGeuAuKqax58C+dF+D65G++orEzhTabNmWIPjrxmtK6MaYYWNMH28XAPwMgOMAvgHg\n/XzYYwC+cOPd8PDw8PDYLDYioY8C+JShhAgBgM9Ya79kjHkZwKeNMf8ZwHMAPn4jHWiw1JlTn5Ym\nS0iZUKTUDn8oXcL+oCBS3DkmQwNF2nRYeuooQrPBGeWqHKmpiR8nNZWyIsUVmCgNlFThCMdCka6v\nc2pc5Ux5iXJPipgQ6a8Iabl7gLSS3buJ/JuviiSzyJkJlxckSrGPCx1MX9WRn0PQaKsq9mGWxt4/\nLNdsl2kuO22V2S5xf5kwVRK6G7KOGEylN83+OeKOsxG2VQ6VZi/1+84+IXn6Byi6s1yRpVcu0n3L\nMeHcUPlSWuzmaJV0HTp3U90P3s6wpqXdFl3xBk2w2Wuwvg129Yu0u6pzhdOujzx2V+hCr6eVkjd3\ngLqqIzl57p3bYKwiL9s8D6HSzNqcDyRW7rWlJmk2TjLXuXaadZbu1ygVl6wR8ev6Een55n7PTkr+\noDZHrOpbsAp66JzzJcjKNTMu22ncVZGDf8pzpU5nXYZCpSHmWQPprwiR7krOuYIsek5DdjHNKQ3Y\n5Wnpio7l++IiZ5cWVR4WXp5JJHO0wKkUoyHpx/6jRHz2c/T3pVdOp/umT1NG2Uj1LX+NvDgbxUa8\nXF4E8NAa7WdA9nQPDw8Pj9sAPlLUw8PDY4dgy5NzOZUwp5IYFR0x0hZV07mZJuwFrRMGJayedVqK\nxIpdCk1NbNF2kqbolO/Z3CyZOmbVNStcGKFXRWFW2Hc9DzLHuOrdABCxShiqWpdNTubkCiTo4zo1\nrtVYU0mM5md47MLm5jkisXGN6MZQqWt9g2QOKpeUH3qTTVDK5NKJnW+68z1Wicb4Wx90pQNlM4JK\nLhWxCl1kE0dPj4pg5CIC5ZyQ2yX2Tc/mRF1t8eYy+83XFcHriNu8Um+zofPZFrU5WGHO0Pe9xaRX\nNqtIrMz6c+mifwNl1sg4U582l3Df3Ax1FW1PIwdV8qp4NTHtIqVdoYtWS+57nU0tcV1FdDIpWlJm\nqUIvqfQdHme7IecI1rCJpP74miB34SBsiiqpGI0q14ZdXBQzoLNY6TWzEmFHzTHX7UxUhLAF9TeE\nShnM2xJVqwhNY7v+AkDCyfdqkSTyk2hvl/5azTdHczfa0je31k2XL3vaST6TCkXl62vCu8KpnIeP\nSqxIwO+qE09/l645JSbTkO+fLlSylgnseuEldA8PD48dAmNvwldhoxgbG7OPP/74Lbueh4eHx07A\nRz/60WestQ+/1nFeQvfw8PDYIfAvdA8PD48dAv9C9/Dw8Ngh8C90Dw8Pjx2CW0qKGmOuAqgCmH6t\nY29zDGF7j2G79x/Y/mPY7v0Htv8YtlP/91trh1/roFv6QgcAY8yxjbC1tzO2+xi2e/+B7T+G7d5/\nYPuPYbv3fy14k4uHh4fHDoF/oXt4eHjsEGzFC/2JLbjmzcZ2H8N27z+w/cew3fsPbP8xbPf+r8It\nt6F7eHh4ePxw4E0uHh4eHjsEt/SFbox51Bhzwhhz2hjzkVt57RuBMWafMeYbxpiXjTEvGWN+jdsH\njDFfNcac4r/9W93Xa4GLfD9njPkS//+gMea7fB/+whiTfa1zbCWMMX3GmM8aY14xxhw3xrx1G96D\nf8dr6AfGmD83xuRv5/tgjPmEMWbKGPMD1bbmnBvCf+dxvGiMecPW9Vywzhj+C6+jF40xf+mqsfG+\n3+QxnDDG/OOt6fXmcMte6Fzx6A8BvBvAvQB+0Rhz7626/g2iA+A3rLX3AngLgF/lPn8EwJPW2iMA\nnuT/3874NVDZQIffBfD71trDAOYAfGhLerVx/AGAv7HW3g3g9aCxbJt7YIzZA+DfAnjYWns/qJbP\nB3F734dPAnh0Rdt6c/5uAEf43+MAPnaL+vha+CRWj+GrAO631r4OwEkAvwkA/Fx/EMB9/Jv/Ybry\n6W4P3EoJ/REAp621Z6y1LQCfBvC+W3j964a1dsJa+yxvL4FeJHtA/f4UH/YpAL+wNT18bRhj9gL4\nWQB/xP83AN4J4LN8yO3e/14AbweXOLTWtqy189hG94ARASgYYyIARQATuI3vg7X2WwBmVzSvN+fv\nA/AnlvAUqID86K3p6fpYawzW2q9YSVL/FKQk8/sAfNpa27TWngVwGtuwItutfKHvAXBR/X+c27YF\njDEHQKX4vgtgxFo7wbuuABhZ52e3A/4bgH8PwGX5HwQwrxb17X4fDgK4CuCP2Wz0R8aYErbRPbDW\nXgLwXwFcAL3IFwA8g+11H4D153y7Ptv/CsD/5e3tOoYueFJ0AzDGlAF8DsCvW2sX9T5LbkK3pauQ\nMebnAExZa5/Z6r5sAhGANwD4mLX2IVDqiC7zyu18DwCAbc3vA32cxgCUsNoUsK1wu8/5a8EY81sg\nk+qfbXVfbiZu5Qv9EoB96v97ue22hjEmA3qZ/5m19vPcPOlUSv47td7vtxg/DuC9xphzIBPXO0H2\n6D5W/YHb/z6MAxi31n6X//9Z0At+u9wDAPhpAGettVettW0Anwfdm+10H4D153xbPdvGmH8B4OcA\n/JIVv+1tNYb1cCtf6F5pGKAAAAF3SURBVE8DOMLMfhZEQHzxFl7/usH25o8DOG6t/T2164sAHuPt\nxwB84Vb3bSOw1v6mtXavtfYAaL6/bq39JQDfAPB+Puy27T8AWGuvALhojLmLm94F4GVsk3vAuADg\nLcaYIq8pN4Ztcx8Y6835FwH8Cnu7vAXAgjLN3FYwxjwKMkG+11pbU7u+COCDxpicMeYgiOD93lb0\ncVOw1t6yfwDeA2KWXwXwW7fy2jfY37eB1MoXATzP/94DskM/CeAUgK8BGNjqvm5gLO8A8CXePgRa\nrKcB/G8Aua3u32v0/UEAx/g+/BWA/u12DwB8FMArAH4A4E8B5G7n+wDgz0H2/jZIS/rQenMOKqn8\nh/xcfx/kzXO7juE0yFbunuf/qY7/LR7DCQDv3ur+38g/Hynq4eHhsUPgSVEPDw+PHQL/Qvfw8PDY\nIfAvdA8PD48dAv9C9/Dw8Ngh8C90Dw8Pjx0C/0L38PDw2CHwL3QPDw+PHQL/Qvfw8PDYIfj/168W\nu0MvjO8AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "TDsB5j-pVYgb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Okay, now let us see what the neural network thinks these examples above are:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "3W8uduFrVYgb",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "outputs = net(images)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "eyv_Amz8VYgd",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The outputs are energies for the 10 classes.\n",
        "Higher the energy for a class, the more the network\n",
        "thinks that the image is of the particular class.\n",
        "So, let's get the index of the highest energy:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "026XSB87VYgd",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "8c9c6f59-9bd4-4d84-8f3a-7045a0840800"
      },
      "cell_type": "code",
      "source": [
        "_, predicted = torch.max(outputs, 1)\n",
        "\n",
        "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n",
        "                              for j in range(4)))"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Predicted:   ship  ship  ship  ship\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "YI8Fk98AVYgf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The results seem pretty good.\n",
        "\n",
        "Let us look at how the network performs on the whole dataset.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "gb1qCPh1VYgg",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "bf686d4b-d86d-4183-f624-682412bf0cfb"
      },
      "cell_type": "code",
      "source": [
        "correct = 0\n",
        "total = 0\n",
        "with torch.no_grad():\n",
        "    for data in testloader:\n",
        "        images, labels = data\n",
        "        outputs = net(images)\n",
        "        _, predicted = torch.max(outputs.data, 1)\n",
        "        total += labels.size(0)\n",
        "        correct += (predicted == labels).sum().item()\n",
        "\n",
        "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
        "    100 * correct / total))"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy of the network on the 10000 test images: 55 %\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "sSOhVeYvVYgj",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "That looks waaay better than chance, which is 10% accuracy (randomly picking\n",
        "a class out of 10 classes).\n",
        "Seems like the network learnt something.\n",
        "\n",
        "Hmmm, what are the classes that performed well, and the classes that did\n",
        "not perform well:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "3Raux5dGVYgk",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        },
        "outputId": "dfac444f-64ff-4e52-b058-b66cf8c6c1b3"
      },
      "cell_type": "code",
      "source": [
        "class_correct = list(0. for i in range(10))\n",
        "class_total = list(0. for i in range(10))\n",
        "with torch.no_grad():\n",
        "    for data in testloader:\n",
        "        images, labels = data\n",
        "        outputs = net(images)\n",
        "        _, predicted = torch.max(outputs, 1)\n",
        "        c = (predicted == labels).squeeze()\n",
        "        for i in range(4):\n",
        "            label = labels[i]\n",
        "            class_correct[label] += c[i].item()\n",
        "            class_total[label] += 1\n",
        "\n",
        "\n",
        "for i in range(10):\n",
        "    print('Accuracy of %5s : %2d %%' % (\n",
        "        classes[i], 100 * class_correct[i] / class_total[i]))"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy of plane : 53 %\n",
            "Accuracy of   car : 60 %\n",
            "Accuracy of  bird : 34 %\n",
            "Accuracy of   cat : 33 %\n",
            "Accuracy of  deer : 47 %\n",
            "Accuracy of   dog : 53 %\n",
            "Accuracy of  frog : 62 %\n",
            "Accuracy of horse : 61 %\n",
            "Accuracy of  ship : 81 %\n",
            "Accuracy of truck : 63 %\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "7pwp_YcGVYgm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Okay, so what next?\n",
        "\n",
        "How do we run these neural networks on the GPU?\n",
        "\n",
        "Training on GPU\n",
        "----------------\n",
        "Just like how you transfer a Tensor on to the GPU, you transfer the neural\n",
        "net onto the GPU.\n",
        "\n",
        "Let's first define our device as the first visible cuda device if we have\n",
        "CUDA available:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "gqpAd1N2VYgn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "4ac9356e-e05e-4c87-8ad6-42cc222eaac9"
      },
      "cell_type": "code",
      "source": [
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "# Assume that we are on a CUDA machine, then this should print a CUDA device:\n",
        "\n",
        "print(device)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "cuda:0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "8LG9_CETVYgo",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The rest of this section assumes that `device` is a CUDA device.\n",
        "\n",
        "Then these methods will recursively go over all modules and convert their\n",
        "parameters and buffers to CUDA tensors:\n",
        "\n",
        "\n",
        " `net.to(device)`\n",
        "\n",
        "\n",
        "Remember that you will have to send the inputs and targets at every step\n",
        "to the GPU too:\n",
        "\n",
        "\n",
        "`inputs, labels = inputs.to(device), labels.to(device)`\n",
        "\n",
        "Why dont I notice MASSIVE speedup compared to CPU? Because your network\n",
        "is realllly small.\n",
        "\n",
        "**Exercise:** Try increasing the width of your network (argument 2 of\n",
        "the first ``nn.Conv2d``, and argument 1 of the second ``nn.Conv2d`` –\n",
        "they need to be the same number), see what kind of speedup you get.\n",
        "\n",
        "**Goals achieved**:\n",
        "\n",
        "- Understanding PyTorch's Tensor library and neural networks at a high level.\n",
        "- Train a small neural network to classify images\n",
        "\n",
        "Training on multiple GPUs\n",
        "-------------------------\n",
        "If you want to see even more MASSIVE speedup using all of your GPUs,\n",
        "please check out :doc:`data_parallel_tutorial`.\n",
        "\n",
        "Where do I go next?\n",
        "-------------------\n",
        "\n",
        "-  `Train neural nets to play video games`\n",
        "-  `Train a state-of-the-art ResNet network on imagenet`\n",
        "-  `Train a face generator using Generative Adversarial Networks`\n",
        "-  `Train a word-level language model using Recurrent LSTM networks`\n",
        "-  `More examples`\n",
        "-  `More tutorials`\n",
        "-  `Discuss PyTorch on the Forums`\n",
        "-  `Chat with other users on Slack`\n",
        "\n"
      ]
    }
  ]
}